-- Logs begin at Wed 2022-08-24 10:27:33 UTC, end at Wed 2022-08-24 11:58:31 UTC. -- Aug 24 11:03:25.126991 dv-cinder-854273-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 11:03:25.137228 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 11:03:25 2022] *** Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: nodename: dv-cinder-854273-2-iscsi Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: machine: x86_64 Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: clock source: unix Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: pcre jit disabled Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: detected number of CPU cores: 4 Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: current working directory: / Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: detected binary path: /usr/bin/uwsgi-core Aug 24 11:03:25.139318 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: your processes number limit is 47680 Aug 24 11:03:25.140048 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: your memory page size is 4096 bytes Aug 24 11:03:25.140048 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: detected max file descriptor number: 2048 Aug 24 11:03:25.140048 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: lock engine: pthread robust mutexes Aug 24 11:03:25.140048 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: thunder lock: enabled Aug 24 11:03:25.140048 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 11:03:25.140048 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 11:03:25.178665 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: Python main interpreter initialized at 0x55b456d399c0 Aug 24 11:03:25.178665 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: python threads support enabled Aug 24 11:03:25.178665 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: your server socket listen backlog is limited to 100 connections Aug 24 11:03:25.178665 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: your mercy for graceful operations on workers is 90 seconds Aug 24 11:03:25.179061 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 11:03:25.179136 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: *** Operational MODE: preforking *** Aug 24 11:03:25.179391 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: *** uWSGI is running in multiple interpreter mode *** Aug 24 11:03:25.179391 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: spawned uWSGI master process (pid: 81853) Aug 24 11:03:25.179601 dv-cinder-854273-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 11:03:25.183645 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: spawned uWSGI worker 1 (pid: 81864, cores: 1) Aug 24 11:03:25.183645 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: spawned uWSGI worker 2 (pid: 81865, cores: 1) Aug 24 11:03:25.183645 dv-cinder-854273-2-iscsi devstack@keystone.service[81853]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 11:03:29.561182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 11:03:29.561182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: from cryptography.utils import int_from_bytes Aug 24 11:03:29.562022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 11:03:29.562022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: from cryptography.utils import int_from_bytes Aug 24 11:03:29.562885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 11:03:29.562885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: from cryptography.utils import int_from_bytes Aug 24 11:03:29.563346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 11:03:29.563346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: from cryptography.utils import int_from_bytes Aug 24 11:03:29.613162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 11:03:29.613162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 11:03:29.613162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] command line args: [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 11:03:29.613474 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 11:03:29.613548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ================================================================================ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 11:03:29.613771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] admin_token = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.614052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] config_dir = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.614135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 11:03:29.614245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.614326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 11:03:29.614431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] config_source = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.614511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] command line args: [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 11:03:29.614613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] debug = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.614724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 11:03:29.614808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.615078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ================================================================================ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 11:03:29.615141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] admin_token = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.615202 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.615368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.615448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] config_dir = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.615560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.615645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.615746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] config_source = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.615844 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.615915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] debug = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.616080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.616195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.616321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_config_append = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.616417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.616516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.616613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.616706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_dir = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.616813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.616909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_file = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_config_append = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_dir = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.617934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_file = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.618018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.618144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.618219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.618333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.618413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.618539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.618615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.618769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.618855 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.619008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.619160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.619160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.619404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.619528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.619658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.619785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.619907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.619998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.620060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.620235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.620365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.620498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.620612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.620690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.83/identity {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.620863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.620942 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] publish_errors = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.621098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.621178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.621333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.621408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.621522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.621748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.621905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.83/identity {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] publish_errors = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] standard_threads = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.622959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_journal = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_json = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] standard_threads = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_stderr = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_syslog = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.623987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.624072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.624164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_journal = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.624299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.624377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_json = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.624499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_stderr = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.624580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.624731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_syslog = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.624809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.625008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:03:29.625091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.625261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.625344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.625490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.625570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.625737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.625819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.626009 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.626595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.626789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.626870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.627032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.627116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.627269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.627346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.627582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.627680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.627752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.627826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.627986 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.628072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.628236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.628322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.628464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.628542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.628705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.628792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.628943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.629020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.629192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.629276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.629427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.629514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.629664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.629740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.629943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.630035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.630243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.630333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.630488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.630570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.630725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.630809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.630959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.631114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.631188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.631382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.631465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.631640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.631718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.631976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.632070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.632141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.632209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.632344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.632429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.632573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.632657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.632809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.632893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.633058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.633136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.633309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.633388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.633537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.633613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.633768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.633845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.634041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.634121 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.634277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.634354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.634513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.634618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.634747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.634859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.634980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.635089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.635222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.635332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.635437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.635539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.635660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.635747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.635878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.635968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.636110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.636193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.636623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.636728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.636790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.636919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.637970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.638095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.638169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.638291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.638376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.638490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.638575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.638728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.638808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.638968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.639048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.639206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.639285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.639442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.639531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.639677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.639767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.639912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.639990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.640153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.640272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.640389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.640476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.640627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.640705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.640857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.640943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.641120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.641210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.641358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.641436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.641595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.642363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.642462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.642532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.642640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.642755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.642882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.643072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.643144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.643317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.643398 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.643571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.643651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.643820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.643899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.644060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.644139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.644295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.644372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.644532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.644609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.644771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.644864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.645010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.645088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.645250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.645330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.645495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.645582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.645736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.645814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.645998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.646082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.646238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.646325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.646475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.646560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.646713 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.646791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.646955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.647043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.647189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.647268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.647424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.647509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.647658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.647744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.647895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.647974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.648145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.648262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.648391 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.648596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.648675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.648848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.648926 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.649078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.649163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.649338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.649421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.649574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.649650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.649805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.649955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.650075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.650157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.external = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.650310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.650387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.650558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.650642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.650796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.650875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.651057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.651148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.password = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.651286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.token = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.651372 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.651531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.651624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.651764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.external = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.651843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.651997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.652076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.652233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.652319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.652486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.652565 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.652734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.password = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.652819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.652980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.653080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.token = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.653202 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.653281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.653450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.653536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.653702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.653792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.653972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.654054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.654211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.654290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.654460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.654539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.654692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.654771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.654929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.655007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.655177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.655257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.655419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.655497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.655663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.655742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.655901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.655982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.656145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.656229 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.656382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.656462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.656638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.656751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.656898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.656979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.657138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.657217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.657406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.657488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.657660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.657739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.657985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.658065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.658236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.658317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.658490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.658570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.658732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.658811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.658991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.659073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.659233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.659319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.659481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.659560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.659720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.659800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.659948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.660027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.660197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.660275 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.660432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.660511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.660686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.660764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.660925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.661007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.661164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.661245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.661415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.661495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.661650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.661729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.661911 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.661995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.662171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.662294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.662490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.662588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.662800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.662882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.663880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.664050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.664135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.664284 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.664366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.664685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.664786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.664862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.664933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.665028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.665108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.665263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.665351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.665516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.665609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.665738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.665819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.665999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.666090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.666244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.666325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.666488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.666567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.666732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.666819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.666965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.667044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.667200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.667279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.667433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.667521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.667669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.667748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.667910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.667989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.668152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.668239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.668385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.668464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.668621 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.668699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.668852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.668937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.669094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.669174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.669328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.669406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.669576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.669707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.669824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.669988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.670065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.670222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.670303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.670482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.670589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.670733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.670813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.670970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.671059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.671207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.671289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.671461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.671540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.671697 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.671776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.671941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.672021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.672181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.672261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.672410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.672488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.672642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.672724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.672883 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.672961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.673164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.673328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.673417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.673572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.673652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.673826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.673934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.674096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.674187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.674338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.674419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.674575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.674739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.674818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.674977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.675058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.675229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.675309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.675495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.675574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.675730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.675811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.675982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.676097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.676233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.676333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.676481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.676563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.676731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.676811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.676980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.677061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.677213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.677295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.677463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.677543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.677703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.677791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.677965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.678049 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.678208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.678288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.678448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.678540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.678698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.678779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.678948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.679026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.679187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.679273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.679423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.679503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.679662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.679742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.679897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.679985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.680133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.680212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.680372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.680451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.680608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.680759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.680842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.680914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.681084 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.681162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.681316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.681403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.681554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.681633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.681791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.681888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.682096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.682192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.682347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.682428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.682586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.682666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.682818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.682905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.683058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.683138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.683303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.683383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.683552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.683638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.683794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.683874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.684028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.684106 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.684263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] role.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.684349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.684519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] role.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.684631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.684756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.684853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.685017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.685107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.685271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.685353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.685523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.685602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.686057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] role.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.688295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.688295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.688430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] role.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.688430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689537 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689889 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.689953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.690957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.691934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.692012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.692169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.692247 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.692426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.692514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.692675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.692753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.692914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.692992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.693150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.693296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.693383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.693455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.693602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.693682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.693838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.693990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.694114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.694195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.694355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.694434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.694592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.694679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.694831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.694910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.695078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.695156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.695327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.695415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.695580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.695661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.695832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.695911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.696072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.696159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.696312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.696391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.696549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.696628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.696790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.696876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.697038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.697143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.697270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.697359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.697517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.697604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.697756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.697836 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.698022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.698103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.698259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.698348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.698574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.698655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.698816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.698896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.699053 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.699147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.699339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.699421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.699582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.699661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.699823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.699903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.700065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.700163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.700314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.700394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.700555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.700634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.700804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.700959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.701036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.701107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.701280 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.701359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.701518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.701606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.701754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.701833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.702014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.702094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.702252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.702343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.702489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.702570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.702663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 11:03:29.702744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.702957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.703195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.703443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.703678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.703920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.704177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:03:29.705032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 11:03:29.732956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.733182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.733414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.734206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.734442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.734668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.742067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.743352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.771465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.771755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.773776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.774034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.833978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.834791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.884315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.884488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.884914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:29.885151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.057204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.057415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.057636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.057856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.058104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.058201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.058322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.058406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.058555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.058773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.058990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.059206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.092431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.092665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.092879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.092972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.093105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.093193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.093327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.093422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.093543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.093625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.093758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.093935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.094020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.094244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.105294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.105615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.118792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.119023 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.119177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.119262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.119414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.119536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.119644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.119736 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.119869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.119953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.120083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.120165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.120294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.120510 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:03:30.147879 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.148026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.148026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 11:03:30.148161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 11:03:30.148347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.148433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.148546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 11:03:30.148629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 11:03:30.148823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.148913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.149088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.149170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.149622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.149707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.149906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.149992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.159849 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.159958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.160125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.160208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.160450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.160542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.160685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.160771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.167722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.167886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.170715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.171012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.171126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.171291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.171390 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.171576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.171668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.171859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.171943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.172129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.172281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.172389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.172466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.172649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.172731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.173489 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.173589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.173903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.174000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.174178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.174278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.174460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.197428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.198362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.198971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.199263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.199906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.200290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.203966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.204350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.204638 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.205193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.205482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.205765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.211043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.211354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.212310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.212571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.215479 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.215773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.216062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.216352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.216641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.216735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.216926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.217017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.217200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.217292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.217474 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.217555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.217827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.218129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.218400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.218667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.235310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.236169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.237373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.237689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.240077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.241607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.243335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.243634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.245006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.245307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.247858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.249562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.251102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.252793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.254355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.254661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.254964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.255255 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.255554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.255850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.256088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.256567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.256854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.257149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.257430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.257720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.271934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.272225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.278248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.278248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.278453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.278453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.278453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.278453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.279648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.281193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.281477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.281755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.283986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.284259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.284766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.285059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.285369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.285669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.285976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.286961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.287251 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.287543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.287835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.288124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.297612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.297977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.298783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.299073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.299364 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.299650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.299935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.300191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.300511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.300792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.301187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.301482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.301763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.302118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.315730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.318158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.318803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.321276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.321692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.323154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.324271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.324644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.325434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.325434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.325770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.327272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.327555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.327854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.331751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.333215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.334501 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.336002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.336252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.336549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.336840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.337127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.338927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.339218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.339504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.339790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.345604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.345927 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.346268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.348380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.348676 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.348955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.353133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.353419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.353708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.354022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.354305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.354600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.354880 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.355152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.355434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.355711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.356010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.356282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.356364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:03:30.356559 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.356655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.356822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.356916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.357140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.357429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.357708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.358013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.358289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.358573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.358882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.359199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.359534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.359878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.360177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:03:30.391897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b456d399c0 pid: 81864 (default app) Aug 24 11:03:30.395464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc3844-23a8-4c06-bf5c-a47835a8b036 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:30.395587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b456d399c0 pid: 81865 (default app) Aug 24 11:03:30.395751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc3844-23a8-4c06-bf5c-a47835a8b036 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:30.396030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc3844-23a8-4c06-bf5c-a47835a8b036 None None] PATH_INFO: `/v3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:30.400343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1/1] 10.222.0.83 () {50 vars in 791 bytes} [Wed Aug 24 11:03:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:38.295477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c157fbf-6523-4893-9bb3-7f41625b8dba None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:38.296082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c157fbf-6523-4893-9bb3-7f41625b8dba None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:38.296082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c157fbf-6523-4893-9bb3-7f41625b8dba None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:38.300912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1/2] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:38.307941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:38.308362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:38.308489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:38.379389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: INFO dbcounter [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] Registered counter for database keystone Aug 24 11:03:38.401213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG oslo_db.sqlalchemy.engines [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 11:03:38.409535 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writer thread running {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 11:03:38.603384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG passlib.handlers.bcrypt [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81864) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 11:03:38.603913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG passlib.handlers.bcrypt [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81864) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 11:03:38.665987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:38.667691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:38.799070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 2/3] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 11:03:39.295710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c157fbf-6523-4893-9bb3-7f41625b8dba None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:39.366024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: INFO dbcounter [None req-8c157fbf-6523-4893-9bb3-7f41625b8dba None None] Registered counter for database keystone Aug 24 11:03:39.385838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG oslo_db.sqlalchemy.engines [None req-8c157fbf-6523-4893-9bb3-7f41625b8dba None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 11:03:39.392389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writer thread running {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 11:03:39.515889 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c157fbf-6523-4893-9bb3-7f41625b8dba None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:39.520933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449a1a3-7310-4752-9057-83ee64bfb626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:39.521572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449a1a3-7310-4752-9057-83ee64bfb626 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:39.521857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449a1a3-7310-4752-9057-83ee64bfb626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:39.522163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449a1a3-7310-4752-9057-83ee64bfb626 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:39.531434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449a1a3-7310-4752-9057-83ee64bfb626 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:39.584628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4449a1a3-7310-4752-9057-83ee64bfb626 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 11:03:39.584882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449a1a3-7310-4752-9057-83ee64bfb626 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:39.595642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 2/4] 10.222.0.83 () {58 vars in 1324 bytes} [Wed Aug 24 11:03:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 11:03:39.602287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:39.617627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884522ec-9c69-4374-a81e-e70eed9995c4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:39.622771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ae618-5b3d-42af-804e-504d0868c89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:39.623919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ae618-5b3d-42af-804e-504d0868c89b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:39.624208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ae618-5b3d-42af-804e-504d0868c89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:39.624481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ae618-5b3d-42af-804e-504d0868c89b admin admin] PATH_INFO: `/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:39.626235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ae618-5b3d-42af-804e-504d0868c89b admin admin] RBAC: Authorizing `identity:get_project(project_id=17f9dd8b207b49df9eb8e79eece61c9b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:39.679767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING keystone.common.rbac_enforcer.enforcer [None req-470ae618-5b3d-42af-804e-504d0868c89b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 11:03:39.680113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ae618-5b3d-42af-804e-504d0868c89b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:39.684942 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 3/5] 10.222.0.83 () {56 vars in 1207 bytes} [Wed Aug 24 11:03:39 2022] GET /identity/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:41.639962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e257e3b-9ec1-46f2-8029-b81d7c30478a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:41.640467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e257e3b-9ec1-46f2-8029-b81d7c30478a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:41.640467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e257e3b-9ec1-46f2-8029-b81d7c30478a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:41.645353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 3/6] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:41.651393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe1071-56f7-48a9-b014-a684fedec676 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:41.651915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe1071-56f7-48a9-b014-a684fedec676 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:41.652031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe1071-56f7-48a9-b014-a684fedec676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:41.672952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-17fe1071-56f7-48a9-b014-a684fedec676 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:41.674561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-17fe1071-56f7-48a9-b014-a684fedec676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:41.683106 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 4/7] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 11:03:41.950962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e257e3b-9ec1-46f2-8029-b81d7c30478a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:41.964828 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e257e3b-9ec1-46f2-8029-b81d7c30478a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:41.969190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930d7bf-d85a-4569-8726-14e79e3eb34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:41.969798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930d7bf-d85a-4569-8726-14e79e3eb34c admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:41.970113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930d7bf-d85a-4569-8726-14e79e3eb34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:41.970389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930d7bf-d85a-4569-8726-14e79e3eb34c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:41.979507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930d7bf-d85a-4569-8726-14e79e3eb34c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:41.981316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930d7bf-d85a-4569-8726-14e79e3eb34c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:41.990851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 4/8] 10.222.0.83 () {58 vars in 1324 bytes} [Wed Aug 24 11:03:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:03:41.996658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe1071-56f7-48a9-b014-a684fedec676 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:42.006686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe1071-56f7-48a9-b014-a684fedec676 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:42.011619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d437870-3b5b-465a-a260-056afd088deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:42.012264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d437870-3b5b-465a-a260-056afd088deb admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:42.012544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d437870-3b5b-465a-a260-056afd088deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:42.012825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d437870-3b5b-465a-a260-056afd088deb admin admin] PATH_INFO: `/v3/users/5d32a6d718b64e35aca00f733d966fd9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:42.014244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d437870-3b5b-465a-a260-056afd088deb admin admin] RBAC: Authorizing `identity:get_user(user_id=5d32a6d718b64e35aca00f733d966fd9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:42.016132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d437870-3b5b-465a-a260-056afd088deb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:42.017735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 5/9] 10.222.0.83 () {56 vars in 1198 bytes} [Wed Aug 24 11:03:41 2022] GET /identity/v3/users/5d32a6d718b64e35aca00f733d966fd9 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:44.088649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46013-6754-42dd-b3a7-477d1f37fbe3 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:44.089450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46013-6754-42dd-b3a7-477d1f37fbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:44.089450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46013-6754-42dd-b3a7-477d1f37fbe3 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:44.090713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 5/10] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:44.096470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f0255-4358-4343-a30d-1103c8c3c736 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:44.096926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f0255-4358-4343-a30d-1103c8c3c736 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:44.097123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f0255-4358-4343-a30d-1103c8c3c736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:44.111033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe4ef8-aa8a-450e-99e0-51f61f95be30 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:44.111310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe4ef8-aa8a-450e-99e0-51f61f95be30 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:44.111583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe4ef8-aa8a-450e-99e0-51f61f95be30 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:44.113418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 6/11] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:03:44.116568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-820f0255-4358-4343-a30d-1103c8c3c736 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:44.118384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-820f0255-4358-4343-a30d-1103c8c3c736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:44.118795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3794a2c-47f5-4ec2-a3bd-35e6ed102eba None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:44.119069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3794a2c-47f5-4ec2-a3bd-35e6ed102eba None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:44.119351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3794a2c-47f5-4ec2-a3bd-35e6ed102eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:44.167663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG passlib.handlers.bcrypt [None req-c3794a2c-47f5-4ec2-a3bd-35e6ed102eba None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81865) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 11:03:44.168068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG passlib.handlers.bcrypt [None req-c3794a2c-47f5-4ec2-a3bd-35e6ed102eba None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81865) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 11:03:44.207947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 6/12] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:03:44.214054 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c3794a2c-47f5-4ec2-a3bd-35e6ed102eba None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:44.215664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c3794a2c-47f5-4ec2-a3bd-35e6ed102eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:44.336766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 7/13] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:03:44.485793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f0255-4358-4343-a30d-1103c8c3c736 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:44.510702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:44.596390 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f0255-4358-4343-a30d-1103c8c3c736 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:44.601423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3794a2c-47f5-4ec2-a3bd-35e6ed102eba None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:44.634247 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d011f8c-e9df-4921-968d-1a3e94c9f925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:44.634912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d011f8c-e9df-4921-968d-1a3e94c9f925 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:44.635201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d011f8c-e9df-4921-968d-1a3e94c9f925 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:44.635473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d011f8c-e9df-4921-968d-1a3e94c9f925 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:44.649999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d011f8c-e9df-4921-968d-1a3e94c9f925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6d011f8c-e9df-4921-968d-1a3e94c9f925 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:03:44.658667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:03:44.659610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:03:44.660470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 7/14] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:03:44.665895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d011f8c-e9df-4921-968d-1a3e94c9f925 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:44.727932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3794a2c-47f5-4ec2-a3bd-35e6ed102eba None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:44.730855 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:44.774145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d53b-eede-472c-8216-5dade14e2bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:44.774731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:44.775016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d53b-eede-472c-8216-5dade14e2bcc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:44.775445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d53b-eede-472c-8216-5dade14e2bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:44.775601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d53b-eede-472c-8216-5dade14e2bcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:44.797796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d011f8c-e9df-4921-968d-1a3e94c9f925 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:44.837254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00532-5865-4d38-850c-74fcd2108d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:44.837894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00532-5865-4d38-850c-74fcd2108d91 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:44.838181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00532-5865-4d38-850c-74fcd2108d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:44.838451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00532-5865-4d38-850c-74fcd2108d91 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:44.839133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00532-5865-4d38-850c-74fcd2108d91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:03:44.839438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00532-5865-4d38-850c-74fcd2108d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:44.840856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00532-5865-4d38-850c-74fcd2108d91 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:44.853705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 8/15] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:03:44.859273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0d53b-eede-472c-8216-5dade14e2bcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:44.859518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00532-5865-4d38-850c-74fcd2108d91 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:44.861054 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0d53b-eede-472c-8216-5dade14e2bcc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:44.967723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00532-5865-4d38-850c-74fcd2108d91 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:44.976202 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 8/16] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:03:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:03:44.981960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d53b-eede-472c-8216-5dade14e2bcc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:44.985052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:45.022592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b3fd9-cb2e-4283-86d4-c4dd90dc9add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:45.023240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b3fd9-cb2e-4283-86d4-c4dd90dc9add None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:45.023516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b3fd9-cb2e-4283-86d4-c4dd90dc9add None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:45.023923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b3fd9-cb2e-4283-86d4-c4dd90dc9add None admin] PATH_INFO: `/v3/users/5d32a6d718b64e35aca00f733d966fd9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:45.025257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b3fd9-cb2e-4283-86d4-c4dd90dc9add None admin] RBAC: Authorizing `identity:get_user(user_id=5d32a6d718b64e35aca00f733d966fd9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:45.026738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b3fd9-cb2e-4283-86d4-c4dd90dc9add None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:45.028398 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 9/17] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:03:44 2022] GET /identity/v3/users/5d32a6d718b64e35aca00f733d966fd9 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:45.034953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b3fd9-cb2e-4283-86d4-c4dd90dc9add None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:45.050871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:45.091179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d53b-eede-472c-8216-5dade14e2bcc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:45.133214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d29a33-f312-4263-a612-80cef17b05fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:45.134132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d29a33-f312-4263-a612-80cef17b05fe None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:45.134415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d29a33-f312-4263-a612-80cef17b05fe None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:45.134699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d29a33-f312-4263-a612-80cef17b05fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:45.148750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b3fd9-cb2e-4283-86d4-c4dd90dc9add None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:45.163599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d29a33-f312-4263-a612-80cef17b05fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:45.165945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d29a33-f312-4263-a612-80cef17b05fe None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:45.194494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b41e51-736b-456f-a8ed-9c8c49d0afee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:45.199545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b41e51-736b-456f-a8ed-9c8c49d0afee None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:45.199545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b41e51-736b-456f-a8ed-9c8c49d0afee None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:45.199545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b41e51-736b-456f-a8ed-9c8c49d0afee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-33d29a33-f312-4263-a612-80cef17b05fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:03:45.199870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:03:45.200707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:03:45.201693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:03:45.201693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:03:45.201693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:03:45.201884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b41e51-736b-456f-a8ed-9c8c49d0afee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:45.201965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 9/18] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:03:45.202059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b41e51-736b-456f-a8ed-9c8c49d0afee None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:45.202059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 10/19] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:03:45.206968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b41e51-736b-456f-a8ed-9c8c49d0afee None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:45.207124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d29a33-f312-4263-a612-80cef17b05fe None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:45.272366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:45.295297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:45.316614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b41e51-736b-456f-a8ed-9c8c49d0afee None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:45.321177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d29a33-f312-4263-a612-80cef17b05fe None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:45.354459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2da98-60c1-47df-8fb9-ff521a4dfd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:45.355072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2da98-60c1-47df-8fb9-ff521a4dfd3c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:45.355352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2da98-60c1-47df-8fb9-ff521a4dfd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:45.355625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2da98-60c1-47df-8fb9-ff521a4dfd3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:45.356368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2da98-60c1-47df-8fb9-ff521a4dfd3c None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.domain.id=default, user.id=5d32a6d718b64e35aca00f733d966fd9) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:03:45.356679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2da98-60c1-47df-8fb9-ff521a4dfd3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:45.357206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c23d2-9707-471b-8c1a-06691bb12c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:45.357832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c23d2-9707-471b-8c1a-06691bb12c61 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:45.358149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c23d2-9707-471b-8c1a-06691bb12c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:45.358256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2da98-60c1-47df-8fb9-ff521a4dfd3c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:45.358425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c23d2-9707-471b-8c1a-06691bb12c61 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:45.359175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c23d2-9707-471b-8c1a-06691bb12c61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:03:45.359474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c23d2-9707-471b-8c1a-06691bb12c61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:45.360886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c23d2-9707-471b-8c1a-06691bb12c61 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:45.366509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 11/20] 10.222.0.83 () {56 vars in 1322 bytes} [Wed Aug 24 11:03:45 2022] GET /identity/v3/role_assignments?user.id=5d32a6d718b64e35aca00f733d966fd9&scope.domain.id=default&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:45.384311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 10/21] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:47.631115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007dacb-1117-4f57-9430-62e301faec95 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:47.631624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007dacb-1117-4f57-9430-62e301faec95 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:47.631624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007dacb-1117-4f57-9430-62e301faec95 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:47.632937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 12/22] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:47.638461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be38e89-ce2a-4568-86f3-6bcb02edead9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:47.638935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be38e89-ce2a-4568-86f3-6bcb02edead9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:47.639027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be38e89-ce2a-4568-86f3-6bcb02edead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:47.666997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3be38e89-ce2a-4568-86f3-6bcb02edead9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:47.668236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3be38e89-ce2a-4568-86f3-6bcb02edead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:47.764241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 11/23] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:03:47.770895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7ce02-714d-42c3-b6e8-01750f513cb6 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:47.771421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7ce02-714d-42c3-b6e8-01750f513cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:47.771920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7ce02-714d-42c3-b6e8-01750f513cb6 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:47.773439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 13/24] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:47.779005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3b399-bc15-4365-a9c7-c11705f93ab6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:47.779295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3b399-bc15-4365-a9c7-c11705f93ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:47.779572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3b399-bc15-4365-a9c7-c11705f93ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:47.817102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-26a3b399-bc15-4365-a9c7-c11705f93ab6 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:47.818365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-26a3b399-bc15-4365-a9c7-c11705f93ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:47.841392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4c98b-6a42-4634-b3ed-c14339a294ac None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:47.842040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4c98b-6a42-4634-b3ed-c14339a294ac None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:47.842531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4c98b-6a42-4634-b3ed-c14339a294ac None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:47.844089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 14/25] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:03:47.850374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb8950-3a54-4b04-afbb-5961b9dae729 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:47.850926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb8950-3a54-4b04-afbb-5961b9dae729 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:47.851395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb8950-3a54-4b04-afbb-5961b9dae729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:47.886895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:47.899491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-59eb8950-3a54-4b04-afbb-5961b9dae729 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:47.905806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-59eb8950-3a54-4b04-afbb-5961b9dae729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:47.944684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 12/26] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 11:03:48.036602 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 15/27] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:03:48.148467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3b399-bc15-4365-a9c7-c11705f93ab6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:48.295125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3b399-bc15-4365-a9c7-c11705f93ab6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:48.352593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168801a3-7253-45de-b8fc-798a46b64a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:48.353258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168801a3-7253-45de-b8fc-798a46b64a0f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:48.353557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168801a3-7253-45de-b8fc-798a46b64a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:48.353843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168801a3-7253-45de-b8fc-798a46b64a0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:48.371885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168801a3-7253-45de-b8fc-798a46b64a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-168801a3-7253-45de-b8fc-798a46b64a0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:03:48.379746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:03:48.380887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:03:48.381630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 13/28] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:03:48.388636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb8950-3a54-4b04-afbb-5961b9dae729 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:48.488536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168801a3-7253-45de-b8fc-798a46b64a0f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:48.532598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:48.554267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:48.612814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb8950-3a54-4b04-afbb-5961b9dae729 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:48.644072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168801a3-7253-45de-b8fc-798a46b64a0f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:48.673948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed5012-f114-477d-ad2e-38c692c354a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:48.675159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed5012-f114-477d-ad2e-38c692c354a7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:48.675648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed5012-f114-477d-ad2e-38c692c354a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:48.675929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed5012-f114-477d-ad2e-38c692c354a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:48.677047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed5012-f114-477d-ad2e-38c692c354a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:03:48.677631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed5012-f114-477d-ad2e-38c692c354a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:48.679356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed5012-f114-477d-ad2e-38c692c354a7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:48.685554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478cf38-4dfd-4bc7-96d6-67e060fcf983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:48.686361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478cf38-4dfd-4bc7-96d6-67e060fcf983 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:48.686676 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478cf38-4dfd-4bc7-96d6-67e060fcf983 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:48.686959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478cf38-4dfd-4bc7-96d6-67e060fcf983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:48.691051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 16/29] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:03:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:48.691994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed5012-f114-477d-ad2e-38c692c354a7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:48.700479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478cf38-4dfd-4bc7-96d6-67e060fcf983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:48.702065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478cf38-4dfd-4bc7-96d6-67e060fcf983 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c478cf38-4dfd-4bc7-96d6-67e060fcf983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:03:48.724237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:03:48.725139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:03:48.725984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:03:48.725984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:03:48.725984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:03:48.725984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 14/30] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:03:48.729896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e174b9f-4d07-4bf5-bb18-cb1ee2953c8b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:48.729896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e174b9f-4d07-4bf5-bb18-cb1ee2953c8b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:48.729896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e174b9f-4d07-4bf5-bb18-cb1ee2953c8b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:48.729896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 15/31] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:48.730216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e174b9f-4d07-4bf5-bb18-cb1ee2953c8b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:48.834109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed5012-f114-477d-ad2e-38c692c354a7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:48.856409 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:48.873389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be1a30-f3b4-4d86-b795-828c6af4548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:48.874214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be1a30-f3b4-4d86-b795-828c6af4548f None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:48.874598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be1a30-f3b4-4d86-b795-828c6af4548f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:48.878269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be1a30-f3b4-4d86-b795-828c6af4548f None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:48.878269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be1a30-f3b4-4d86-b795-828c6af4548f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:48.878269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be1a30-f3b4-4d86-b795-828c6af4548f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:48.882453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e174b9f-4d07-4bf5-bb18-cb1ee2953c8b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:48.886619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:48.932977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb85f3c-a12e-4ea7-98dc-0b3e7f6c6d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:48.936812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb85f3c-a12e-4ea7-98dc-0b3e7f6c6d3c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:48.936812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb85f3c-a12e-4ea7-98dc-0b3e7f6c6d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:48.936812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb85f3c-a12e-4ea7-98dc-0b3e7f6c6d3c None admin] PATH_INFO: `/v3/users/5d32a6d718b64e35aca00f733d966fd9` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:48.938596 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb85f3c-a12e-4ea7-98dc-0b3e7f6c6d3c None admin] RBAC: Authorizing `identity:get_user(user_id=5d32a6d718b64e35aca00f733d966fd9)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:48.940305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb85f3c-a12e-4ea7-98dc-0b3e7f6c6d3c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:48.947885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 16/32] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:03:48 2022] GET /identity/v3/users/5d32a6d718b64e35aca00f733d966fd9 => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:48.947885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb85f3c-a12e-4ea7-98dc-0b3e7f6c6d3c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:49.133603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb85f3c-a12e-4ea7-98dc-0b3e7f6c6d3c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:49.187115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3049416-7451-40a1-a785-884abd5d3da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:49.188111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3049416-7451-40a1-a785-884abd5d3da6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:49.188693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3049416-7451-40a1-a785-884abd5d3da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:49.189171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3049416-7451-40a1-a785-884abd5d3da6 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:49.190932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3049416-7451-40a1-a785-884abd5d3da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:03:49.191563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3049416-7451-40a1-a785-884abd5d3da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:49.193634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3049416-7451-40a1-a785-884abd5d3da6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:49.198045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 17/33] 10.222.0.83 () {60 vars in 1131 bytes} [Wed Aug 24 11:03:48 2022] POST /identity/v3/roles => generated 219 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:03:49.200068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a255f-b97f-446f-a0a7-bef0627572de None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:49.200375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a255f-b97f-446f-a0a7-bef0627572de None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:49.200933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a255f-b97f-446f-a0a7-bef0627572de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:49.214946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 17/34] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:49.216243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3049416-7451-40a1-a785-884abd5d3da6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:49.235674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-504a255f-b97f-446f-a0a7-bef0627572de None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:49.239613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-504a255f-b97f-446f-a0a7-bef0627572de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:49.276302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:49.393696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 18/35] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:03:49.396032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a255f-b97f-446f-a0a7-bef0627572de None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:49.537030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3049416-7451-40a1-a785-884abd5d3da6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:49.598739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a255f-b97f-446f-a0a7-bef0627572de None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:49.601610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2339f-b880-4ed7-8307-25cb56432249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:49.602310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2339f-b880-4ed7-8307-25cb56432249 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:49.602592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2339f-b880-4ed7-8307-25cb56432249 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:49.602865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2339f-b880-4ed7-8307-25cb56432249 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:49.604018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2339f-b880-4ed7-8307-25cb56432249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:49.605482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2339f-b880-4ed7-8307-25cb56432249 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:49.607112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 18/36] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:03:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:49.609091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339794c6-a335-4231-8daa-9db256dc66a1 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:49.609384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339794c6-a335-4231-8daa-9db256dc66a1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:49.609654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339794c6-a335-4231-8daa-9db256dc66a1 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:49.611180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 19/37] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:49.618696 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339794c6-a335-4231-8daa-9db256dc66a1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:49.664084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:49.677936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec42ba-59c0-464f-92c9-dce168d7b223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:49.679518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec42ba-59c0-464f-92c9-dce168d7b223 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:49.679518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec42ba-59c0-464f-92c9-dce168d7b223 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:49.680365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec42ba-59c0-464f-92c9-dce168d7b223 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:49.683945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec42ba-59c0-464f-92c9-dce168d7b223 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:49.684144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec42ba-59c0-464f-92c9-dce168d7b223 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:49.739477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:49.759217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339794c6-a335-4231-8daa-9db256dc66a1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:49.799540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7850568-871f-4229-b093-12a61424e87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:49.800208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7850568-871f-4229-b093-12a61424e87e None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:49.800571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7850568-871f-4229-b093-12a61424e87e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:49.800855 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7850568-871f-4229-b093-12a61424e87e None admin] PATH_INFO: `/v3/domains/default/users/5d32a6d718b64e35aca00f733d966fd9/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:49.802880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7850568-871f-4229-b093-12a61424e87e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5d32a6d718b64e35aca00f733d966fd9, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:49.804428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7850568-871f-4229-b093-12a61424e87e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:49.833927 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 19/38] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:03:49 2022] POST /identity/v3/projects => generated 301 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:03:49.835230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a02046-7515-49aa-9d07-902bb6ffd83d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:49.835523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a02046-7515-49aa-9d07-902bb6ffd83d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:49.835807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a02046-7515-49aa-9d07-902bb6ffd83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:49.858546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d1a02046-7515-49aa-9d07-902bb6ffd83d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:49.859850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d1a02046-7515-49aa-9d07-902bb6ffd83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:49.954289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 20/39] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:03:49.955534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fe43e-95b5-4ac1-9727-d49bb01bdde0 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:49.955886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fe43e-95b5-4ac1-9727-d49bb01bdde0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:49.956157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fe43e-95b5-4ac1-9727-d49bb01bdde0 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:49.961945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 21/40] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:49.961945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fe43e-95b5-4ac1-9727-d49bb01bdde0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:50.070996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 20/41] 10.222.0.83 () {58 vars in 1361 bytes} [Wed Aug 24 11:03:49 2022] PUT /identity/v3/domains/default/users/5d32a6d718b64e35aca00f733d966fd9/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:03:50.072863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fcb9d-db19-417d-99ed-da11e1a0421a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:50.073155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fcb9d-db19-417d-99ed-da11e1a0421a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:50.073434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fcb9d-db19-417d-99ed-da11e1a0421a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:50.106243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f77fcb9d-db19-417d-99ed-da11e1a0421a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:50.107956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f77fcb9d-db19-417d-99ed-da11e1a0421a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:50.129036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fe43e-95b5-4ac1-9727-d49bb01bdde0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:50.164632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:50.196667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a3144-e4d1-4021-bae1-71bb4561f787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:50.197312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a3144-e4d1-4021-bae1-71bb4561f787 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:50.197711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a3144-e4d1-4021-bae1-71bb4561f787 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:50.198118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a3144-e4d1-4021-bae1-71bb4561f787 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:50.198955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a3144-e4d1-4021-bae1-71bb4561f787 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:50.200422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a3144-e4d1-4021-bae1-71bb4561f787 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:50.238366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 21/42] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:03:50.291769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fcb9d-db19-417d-99ed-da11e1a0421a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:50.313928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 22/43] 10.222.0.83 () {60 vars in 1131 bytes} [Wed Aug 24 11:03:49 2022] POST /identity/v3/roles => generated 225 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:03:50.402479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:50.455750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fcb9d-db19-417d-99ed-da11e1a0421a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:50.512263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0493d-ed54-4545-baec-a709ba8d3113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:50.512885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0493d-ed54-4545-baec-a709ba8d3113 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:50.513172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0493d-ed54-4545-baec-a709ba8d3113 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:50.513446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0493d-ed54-4545-baec-a709ba8d3113 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:50.514686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0493d-ed54-4545-baec-a709ba8d3113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:50.516373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0493d-ed54-4545-baec-a709ba8d3113 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:50.521499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 22/44] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:03:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:03:50.526931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a3144-e4d1-4021-bae1-71bb4561f787 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:50.556705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0493d-ed54-4545-baec-a709ba8d3113 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:50.664048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a3144-e4d1-4021-bae1-71bb4561f787 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:50.697920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0493d-ed54-4545-baec-a709ba8d3113 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:50.708376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805043b-b811-4ca8-896a-4b6bde81de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:50.709127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805043b-b811-4ca8-896a-4b6bde81de53 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:50.709415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805043b-b811-4ca8-896a-4b6bde81de53 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:50.709804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805043b-b811-4ca8-896a-4b6bde81de53 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:50.710532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805043b-b811-4ca8-896a-4b6bde81de53 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:50.712041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805043b-b811-4ca8-896a-4b6bde81de53 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:50.748177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a274e-84f3-49a7-909f-69183baa9ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:50.748947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a274e-84f3-49a7-909f-69183baa9ac5 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:50.759356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a274e-84f3-49a7-909f-69183baa9ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:50.759356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a274e-84f3-49a7-909f-69183baa9ac5 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:50.759356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a274e-84f3-49a7-909f-69183baa9ac5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:50.759356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a274e-84f3-49a7-909f-69183baa9ac5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:50.833904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 23/45] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:03:50 2022] POST /identity/v3/projects => generated 312 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:03:50.858433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 23/46] 10.222.0.83 () {60 vars in 1131 bytes} [Wed Aug 24 11:03:50 2022] POST /identity/v3/roles => generated 223 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:03:52.104046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054df184-4dbb-4d3b-bc20-f3a4e9e09c5f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:52.104576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054df184-4dbb-4d3b-bc20-f3a4e9e09c5f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:52.104681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054df184-4dbb-4d3b-bc20-f3a4e9e09c5f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:52.106195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 24/47] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:52.115567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330544d-a1aa-4dcd-81ab-153d27f3aa32 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:52.116081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330544d-a1aa-4dcd-81ab-153d27f3aa32 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:52.116162 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330544d-a1aa-4dcd-81ab-153d27f3aa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:52.122991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:52.136062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d330544d-a1aa-4dcd-81ab-153d27f3aa32 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:52.137303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d330544d-a1aa-4dcd-81ab-153d27f3aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:52.233422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 24/48] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:03:52.507961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054df184-4dbb-4d3b-bc20-f3a4e9e09c5f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:52.532021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:52.637446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054df184-4dbb-4d3b-bc20-f3a4e9e09c5f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:52.782887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3718cc7-5f13-4e41-bb11-02cba68bd000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:52.783450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3718cc7-5f13-4e41-bb11-02cba68bd000 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:52.783759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3718cc7-5f13-4e41-bb11-02cba68bd000 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:52.784034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3718cc7-5f13-4e41-bb11-02cba68bd000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:52.795643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3718cc7-5f13-4e41-bb11-02cba68bd000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e3718cc7-5f13-4e41-bb11-02cba68bd000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:03:52.800908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:03:52.801809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:03:52.802932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 25/49] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:03:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:03:52.808602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330544d-a1aa-4dcd-81ab-153d27f3aa32 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:52.828301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc04910-f03c-46a4-9473-c2acf69ad60d None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:52.828574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc04910-f03c-46a4-9473-c2acf69ad60d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:52.828839 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc04910-f03c-46a4-9473-c2acf69ad60d None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:52.830239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 26/50] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:03:52.836956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1e2e4-a413-46e4-86d9-4b2c154c4d70 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:52.837252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1e2e4-a413-46e4-86d9-4b2c154c4d70 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:52.837513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1e2e4-a413-46e4-86d9-4b2c154c4d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:52.856804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d4a1e2e4-a413-46e4-86d9-4b2c154c4d70 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:52.858063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d4a1e2e4-a413-46e4-86d9-4b2c154c4d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:52.926244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330544d-a1aa-4dcd-81ab-153d27f3aa32 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:52.934637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 27/51] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:03:52.962626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:52.968291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173ea2f-76da-4a78-826b-9dc4b8553778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:52.968960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173ea2f-76da-4a78-826b-9dc4b8553778 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:52.969190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173ea2f-76da-4a78-826b-9dc4b8553778 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:52.969468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173ea2f-76da-4a78-826b-9dc4b8553778 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:52.970196 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173ea2f-76da-4a78-826b-9dc4b8553778 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:03:52.970556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173ea2f-76da-4a78-826b-9dc4b8553778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:52.972476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173ea2f-76da-4a78-826b-9dc4b8553778 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:53.092322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 25/52] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:03:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:53.099005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1e2e4-a413-46e4-86d9-4b2c154c4d70 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:53.112685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:53.208264 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173ea2f-76da-4a78-826b-9dc4b8553778 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:53.218046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1e2e4-a413-46e4-86d9-4b2c154c4d70 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:53.257360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc52e4c-8a15-4bf3-be03-0974f052c933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:53.258052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc52e4c-8a15-4bf3-be03-0974f052c933 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:53.258340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc52e4c-8a15-4bf3-be03-0974f052c933 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:53.258733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc52e4c-8a15-4bf3-be03-0974f052c933 None admin] PATH_INFO: `/v3/users/5d32a6d718b64e35aca00f733d966fd9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:53.259942 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc52e4c-8a15-4bf3-be03-0974f052c933 None admin] RBAC: Authorizing `identity:get_user(user_id=5d32a6d718b64e35aca00f733d966fd9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:53.261264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc52e4c-8a15-4bf3-be03-0974f052c933 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:53.262915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 28/53] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:03:53 2022] GET /identity/v3/users/5d32a6d718b64e35aca00f733d966fd9 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:53.272776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc52e4c-8a15-4bf3-be03-0974f052c933 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:53.322255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173ea2f-76da-4a78-826b-9dc4b8553778 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:53.363094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f186e-0923-4cb1-85dc-3e5767a66332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:53.363652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f186e-0923-4cb1-85dc-3e5767a66332 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:53.363930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f186e-0923-4cb1-85dc-3e5767a66332 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:53.364204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f186e-0923-4cb1-85dc-3e5767a66332 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:53.365378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f186e-0923-4cb1-85dc-3e5767a66332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:53.366889 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f186e-0923-4cb1-85dc-3e5767a66332 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:53.368590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 26/54] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:03:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:03:53.373264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:53.374070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f186e-0923-4cb1-85dc-3e5767a66332 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:53.395338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc52e4c-8a15-4bf3-be03-0974f052c933 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:53.440761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a219d90-27e7-4742-bec2-86dd05ea28ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:53.441599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a219d90-27e7-4742-bec2-86dd05ea28ec None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:53.442031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a219d90-27e7-4742-bec2-86dd05ea28ec None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:53.442457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a219d90-27e7-4742-bec2-86dd05ea28ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:53.443750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a219d90-27e7-4742-bec2-86dd05ea28ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:53.445301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a219d90-27e7-4742-bec2-86dd05ea28ec None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:53.446956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 29/55] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:03:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:03:53.451048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:53.453186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a219d90-27e7-4742-bec2-86dd05ea28ec None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:53.500551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f186e-0923-4cb1-85dc-3e5767a66332 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:53.545572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f717666-ce32-4a23-9d80-474f132eb860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:53.546228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f717666-ce32-4a23-9d80-474f132eb860 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:53.546509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f717666-ce32-4a23-9d80-474f132eb860 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:53.547058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f717666-ce32-4a23-9d80-474f132eb860 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:53.548054 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f717666-ce32-4a23-9d80-474f132eb860 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:53.550310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f717666-ce32-4a23-9d80-474f132eb860 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:53.577323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a219d90-27e7-4742-bec2-86dd05ea28ec None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:53.613286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0bbba-9b1d-42ec-a7f3-cd3a1d336098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:53.613908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0bbba-9b1d-42ec-a7f3-cd3a1d336098 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:53.614200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0bbba-9b1d-42ec-a7f3-cd3a1d336098 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:53.614477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0bbba-9b1d-42ec-a7f3-cd3a1d336098 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:53.615225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0bbba-9b1d-42ec-a7f3-cd3a1d336098 None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.domain.id=default, user.id=5d32a6d718b64e35aca00f733d966fd9) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:03:53.615534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0bbba-9b1d-42ec-a7f3-cd3a1d336098 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:53.617042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0bbba-9b1d-42ec-a7f3-cd3a1d336098 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:53.623904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 30/56] 10.222.0.83 () {56 vars in 1322 bytes} [Wed Aug 24 11:03:53 2022] GET /identity/v3/role_assignments?user.id=5d32a6d718b64e35aca00f733d966fd9&scope.domain.id=default&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 530 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:03:53.647928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 27/57] 10.222.0.83 () {60 vars in 1140 bytes} [Wed Aug 24 11:03:53 2022] POST /identity/v3/projects => generated 298 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:03:55.652231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cac99-d982-4458-bc22-b2ed5711dad8 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:55.652785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cac99-d982-4458-bc22-b2ed5711dad8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:55.652785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cac99-d982-4458-bc22-b2ed5711dad8 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:55.654110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 31/58] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:55.660716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445a1fd-6f8a-47a9-801e-2b1f989e13bc None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:55.661204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445a1fd-6f8a-47a9-801e-2b1f989e13bc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:55.661316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445a1fd-6f8a-47a9-801e-2b1f989e13bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:55.680442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f445a1fd-6f8a-47a9-801e-2b1f989e13bc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:55.681677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f445a1fd-6f8a-47a9-801e-2b1f989e13bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:55.759268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 28/59] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:03:56.040461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cac99-d982-4458-bc22-b2ed5711dad8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:56.054386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:56.155250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cac99-d982-4458-bc22-b2ed5711dad8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:56.197744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1fa29-8504-42fd-89a7-87d2d1fc590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:56.198422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1fa29-8504-42fd-89a7-87d2d1fc590a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:56.198714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1fa29-8504-42fd-89a7-87d2d1fc590a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:56.198993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1fa29-8504-42fd-89a7-87d2d1fc590a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:56.200196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1fa29-8504-42fd-89a7-87d2d1fc590a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:56.201740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1fa29-8504-42fd-89a7-87d2d1fc590a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:56.203630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 32/60] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:03:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:03:56.210401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445a1fd-6f8a-47a9-801e-2b1f989e13bc None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:03:56.270720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:03:56.338071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445a1fd-6f8a-47a9-801e-2b1f989e13bc None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:03:56.378930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afce0b-2492-488f-8068-8bfa7b3e5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:03:56.379723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afce0b-2492-488f-8068-8bfa7b3e5abe None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:56.381029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afce0b-2492-488f-8068-8bfa7b3e5abe None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:56.382365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afce0b-2492-488f-8068-8bfa7b3e5abe None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:56.383178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afce0b-2492-488f-8068-8bfa7b3e5abe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:03:56.390244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afce0b-2492-488f-8068-8bfa7b3e5abe None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:03:56.518945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 29/61] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:03:56 2022] POST /identity/v3/users => generated 275 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:03:59.891313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb1359-c1ab-4c5c-9cb0-a1ed75fc0181 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:59.891313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb1359-c1ab-4c5c-9cb0-a1ed75fc0181 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:59.891313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb1359-c1ab-4c5c-9cb0-a1ed75fc0181 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:59.892517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 33/62] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:59.896928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e163db-8371-45ad-9eb2-e6b2e3cbcbc9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:59.897793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e163db-8371-45ad-9eb2-e6b2e3cbcbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:59.898291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e163db-8371-45ad-9eb2-e6b2e3cbcbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:59.902626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce265d-0f48-49c5-a274-6f918a2f5725 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:59.903599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce265d-0f48-49c5-a274-6f918a2f5725 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:59.904649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce265d-0f48-49c5-a274-6f918a2f5725 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:59.909262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 34/63] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:03:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:03:59.912100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d64840-add9-4212-a17c-92e71bb5d036 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:03:59.912662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d64840-add9-4212-a17c-92e71bb5d036 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:03:59.913149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d64840-add9-4212-a17c-92e71bb5d036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:03:59.929711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f7e163db-8371-45ad-9eb2-e6b2e3cbcbc9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:59.931952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f7e163db-8371-45ad-9eb2-e6b2e3cbcbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:03:59.948649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b6d64840-add9-4212-a17c-92e71bb5d036 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:03:59.949901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b6d64840-add9-4212-a17c-92e71bb5d036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:00.088240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 30/64] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:00.112706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 35/65] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:00.208552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1d4e-b3bc-4d7b-919a-c3f2b6d487b5 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:00.208860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1d4e-b3bc-4d7b-919a-c3f2b6d487b5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:00.209023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1d4e-b3bc-4d7b-919a-c3f2b6d487b5 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:00.210578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 31/66] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:00.216053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0bb7-ef6b-42cb-bc59-a47de7fc014b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:00.216341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0bb7-ef6b-42cb-bc59-a47de7fc014b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:00.216619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0bb7-ef6b-42cb-bc59-a47de7fc014b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:00.241193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6f4c0bb7-ef6b-42cb-bc59-a47de7fc014b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:00.241971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6f4c0bb7-ef6b-42cb-bc59-a47de7fc014b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:00.244173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:00.320621 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b21c4-440c-4c10-9763-7202544dd6a6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:00.321030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b21c4-440c-4c10-9763-7202544dd6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:00.321177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b21c4-440c-4c10-9763-7202544dd6a6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:00.322553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 32/67] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:04:00.330922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04564555-06ea-4ac4-8139-366f82c5a032 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:00.331325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04564555-06ea-4ac4-8139-366f82c5a032 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:00.331566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04564555-06ea-4ac4-8139-366f82c5a032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:00.357127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-04564555-06ea-4ac4-8139-366f82c5a032 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:00.358390 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-04564555-06ea-4ac4-8139-366f82c5a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:00.391047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 36/68] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:00.405212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:00.439130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c0bb7-ef6b-42cb-bc59-a47de7fc014b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:00.489157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 33/69] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:00.542064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04564555-06ea-4ac4-8139-366f82c5a032 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:00.579708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c0bb7-ef6b-42cb-bc59-a47de7fc014b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:00.624968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f107025-9744-48ad-be37-b8aeab76479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:00.626311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f107025-9744-48ad-be37-b8aeab76479c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:00.626819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f107025-9744-48ad-be37-b8aeab76479c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:00.627232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f107025-9744-48ad-be37-b8aeab76479c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:00.639725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f107025-9744-48ad-be37-b8aeab76479c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:00.646086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-2f107025-9744-48ad-be37-b8aeab76479c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:00.648482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:00.649606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:00.650556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:00.653211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 37/70] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:04:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:04:00.656580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965b5b-424b-41af-beab-ed83eea71467 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:00.657031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965b5b-424b-41af-beab-ed83eea71467 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:00.657446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965b5b-424b-41af-beab-ed83eea71467 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:00.659099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 38/71] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:00.667563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7ed01-2ba9-4049-a404-f9e94973056a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:00.667851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7ed01-2ba9-4049-a404-f9e94973056a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:00.668127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7ed01-2ba9-4049-a404-f9e94973056a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:00.687532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a4d7ed01-2ba9-4049-a404-f9e94973056a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:00.688823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a4d7ed01-2ba9-4049-a404-f9e94973056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:00.703643 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04564555-06ea-4ac4-8139-366f82c5a032 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:00.754472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ec989-b1c9-4d4d-bb0d-67f91ebb48d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:00.756034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ec989-b1c9-4d4d-bb0d-67f91ebb48d4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:00.756528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ec989-b1c9-4d4d-bb0d-67f91ebb48d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:00.757349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ec989-b1c9-4d4d-bb0d-67f91ebb48d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:00.763502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 39/72] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:00.770949 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7ed01-2ba9-4049-a404-f9e94973056a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:00.781422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ec989-b1c9-4d4d-bb0d-67f91ebb48d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-270ec989-b1c9-4d4d-bb0d-67f91ebb48d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:00.793115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:00.794450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:00.795191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:00.796289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 34/73] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:00.797404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ec989-b1c9-4d4d-bb0d-67f91ebb48d4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:00.834309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:00.932828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7ed01-2ba9-4049-a404-f9e94973056a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:00.963797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ec989-b1c9-4d4d-bb0d-67f91ebb48d4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:00.980673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:00.991758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518825c-7acf-44b8-8b86-e36875171d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:00.992513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518825c-7acf-44b8-8b86-e36875171d75 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:00.992905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518825c-7acf-44b8-8b86-e36875171d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:00.993242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518825c-7acf-44b8-8b86-e36875171d75 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:00.993990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518825c-7acf-44b8-8b86-e36875171d75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:00.994300 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518825c-7acf-44b8-8b86-e36875171d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:00.995978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518825c-7acf-44b8-8b86-e36875171d75 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:01.015129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 40/74] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:01.015902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518825c-7acf-44b8-8b86-e36875171d75 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.024331 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f8778-d2c3-4249-872b-28c337874643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.024987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f8778-d2c3-4249-872b-28c337874643 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.025339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f8778-d2c3-4249-872b-28c337874643 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.025627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f8778-d2c3-4249-872b-28c337874643 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.039691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f8778-d2c3-4249-872b-28c337874643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-034f8778-d2c3-4249-872b-28c337874643 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:01.044833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:01.045805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 35/75] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:01.052762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f8778-d2c3-4249-872b-28c337874643 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.142354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:01.146305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518825c-7acf-44b8-8b86-e36875171d75 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.162937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f8778-d2c3-4249-872b-28c337874643 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.185145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df955e8-fc43-4457-aec0-a59ace97fca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.185780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df955e8-fc43-4457-aec0-a59ace97fca9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.186093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df955e8-fc43-4457-aec0-a59ace97fca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.186376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df955e8-fc43-4457-aec0-a59ace97fca9 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.187063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df955e8-fc43-4457-aec0-a59ace97fca9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:01.187371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df955e8-fc43-4457-aec0-a59ace97fca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:01.188791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df955e8-fc43-4457-aec0-a59ace97fca9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:01.198742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 41/76] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:01.199611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df955e8-fc43-4457-aec0-a59ace97fca9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.204476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c398ef0-dbaa-44c1-a127-70a6ddbdb8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.205136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c398ef0-dbaa-44c1-a127-70a6ddbdb8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.205449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c398ef0-dbaa-44c1-a127-70a6ddbdb8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.205755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c398ef0-dbaa-44c1-a127-70a6ddbdb8a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.216543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c398ef0-dbaa-44c1-a127-70a6ddbdb8a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0c398ef0-dbaa-44c1-a127-70a6ddbdb8a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:01.221386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:01.222655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 36/77] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:01.223507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c398ef0-dbaa-44c1-a127-70a6ddbdb8a2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.259016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:01.311641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df955e8-fc43-4457-aec0-a59ace97fca9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.342272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c398ef0-dbaa-44c1-a127-70a6ddbdb8a2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.349399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db82f4-9fa0-48be-8cb9-88b56f2e22bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.350071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db82f4-9fa0-48be-8cb9-88b56f2e22bd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.350360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db82f4-9fa0-48be-8cb9-88b56f2e22bd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.350751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db82f4-9fa0-48be-8cb9-88b56f2e22bd None admin] PATH_INFO: `/v3/users/5d32a6d718b64e35aca00f733d966fd9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.351848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db82f4-9fa0-48be-8cb9-88b56f2e22bd None admin] RBAC: Authorizing `identity:get_user(user_id=5d32a6d718b64e35aca00f733d966fd9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:01.353318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db82f4-9fa0-48be-8cb9-88b56f2e22bd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:01.354992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 42/78] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/users/5d32a6d718b64e35aca00f733d966fd9 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:01.355781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db82f4-9fa0-48be-8cb9-88b56f2e22bd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.383649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b9312-6e4d-497e-b890-578b4f5a913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.384280 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b9312-6e4d-497e-b890-578b4f5a913a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.384601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b9312-6e4d-497e-b890-578b4f5a913a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.384921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b9312-6e4d-497e-b890-578b4f5a913a None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.385644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b9312-6e4d-497e-b890-578b4f5a913a None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:01.386017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b9312-6e4d-497e-b890-578b4f5a913a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:01.387458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b9312-6e4d-497e-b890-578b4f5a913a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:01.396051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 37/79] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:01.396896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b9312-6e4d-497e-b890-578b4f5a913a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.399990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:01.469575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db82f4-9fa0-48be-8cb9-88b56f2e22bd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.491238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:01.505830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981b888-f48d-44bf-b14a-0c04413f2ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.506637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981b888-f48d-44bf-b14a-0c04413f2ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.506925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981b888-f48d-44bf-b14a-0c04413f2ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.507198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b9312-6e4d-497e-b890-578b4f5a913a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.507304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981b888-f48d-44bf-b14a-0c04413f2ca5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.508384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981b888-f48d-44bf-b14a-0c04413f2ca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:01.509863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981b888-f48d-44bf-b14a-0c04413f2ca5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:01.511589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 43/80] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:01.512382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981b888-f48d-44bf-b14a-0c04413f2ca5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.544293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7a7a-7c68-429a-b281-4924a56810c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.544952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7a7a-7c68-429a-b281-4924a56810c5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.545298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7a7a-7c68-429a-b281-4924a56810c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.545683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7a7a-7c68-429a-b281-4924a56810c5 None admin] PATH_INFO: `/v3/users/e22fbb4f1849455e917f7324ffd888a3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.562342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b7a7a-7c68-429a-b281-4924a56810c5 None admin] RBAC: Authorizing `identity:get_user(user_id=e22fbb4f1849455e917f7324ffd888a3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:01.563836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b7a7a-7c68-429a-b281-4924a56810c5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:01.566316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 38/81] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/users/e22fbb4f1849455e917f7324ffd888a3 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:01.567643 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7a7a-7c68-429a-b281-4924a56810c5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.621328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981b888-f48d-44bf-b14a-0c04413f2ca5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.628654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:01.663433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbfef2-5fa2-4b9b-89b0-13daf1077b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.664058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbfef2-5fa2-4b9b-89b0-13daf1077b6c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.664349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbfef2-5fa2-4b9b-89b0-13daf1077b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.664623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbfef2-5fa2-4b9b-89b0-13daf1077b6c None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.665321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbfef2-5fa2-4b9b-89b0-13daf1077b6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:01.665638 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbfef2-5fa2-4b9b-89b0-13daf1077b6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:01.667099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbfef2-5fa2-4b9b-89b0-13daf1077b6c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:01.673483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7a7a-7c68-429a-b281-4924a56810c5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.676058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 44/82] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:01.676929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbfef2-5fa2-4b9b-89b0-13daf1077b6c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.717472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c650bb-4497-44af-9151-9ddeebc45f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.718132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c650bb-4497-44af-9151-9ddeebc45f8c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.718411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c650bb-4497-44af-9151-9ddeebc45f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.718683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c650bb-4497-44af-9151-9ddeebc45f8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.734920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:01.792494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbfef2-5fa2-4b9b-89b0-13daf1077b6c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.796726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c650bb-4497-44af-9151-9ddeebc45f8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:01.798262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c650bb-4497-44af-9151-9ddeebc45f8c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:01.830744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea34a8-d8d6-434c-ae12-79f024b44a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.831987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea34a8-d8d6-434c-ae12-79f024b44a3a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.832251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea34a8-d8d6-434c-ae12-79f024b44a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.832525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea34a8-d8d6-434c-ae12-79f024b44a3a None admin] PATH_INFO: `/v3/users/e22fbb4f1849455e917f7324ffd888a3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.833777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea34a8-d8d6-434c-ae12-79f024b44a3a None admin] RBAC: Authorizing `identity:get_user(user_id=e22fbb4f1849455e917f7324ffd888a3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:01.835305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea34a8-d8d6-434c-ae12-79f024b44a3a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:01.837317 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 45/83] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/users/e22fbb4f1849455e917f7324ffd888a3 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:01.837758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea34a8-d8d6-434c-ae12-79f024b44a3a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.864106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:01.906101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 39/84] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:01.907302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c650bb-4497-44af-9151-9ddeebc45f8c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:01.946802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea34a8-d8d6-434c-ae12-79f024b44a3a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:01.965023 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:01.982602 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdda825-cdb5-4b0b-b0eb-8ac4168280c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:01.983221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdda825-cdb5-4b0b-b0eb-8ac4168280c7 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:01.983509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdda825-cdb5-4b0b-b0eb-8ac4168280c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:01.983843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdda825-cdb5-4b0b-b0eb-8ac4168280c7 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:01.984446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdda825-cdb5-4b0b-b0eb-8ac4168280c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:01.986055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdda825-cdb5-4b0b-b0eb-8ac4168280c7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:02.013292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c650bb-4497-44af-9151-9ddeebc45f8c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:02.054221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e159b-fb2e-4b6d-bbf2-bc5cf9b9e293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:02.054814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e159b-fb2e-4b6d-bbf2-bc5cf9b9e293 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:02.055111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e159b-fb2e-4b6d-bbf2-bc5cf9b9e293 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:02.055373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e159b-fb2e-4b6d-bbf2-bc5cf9b9e293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:02.088322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:02.107547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 46/85] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:01 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:02.108840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdda825-cdb5-4b0b-b0eb-8ac4168280c7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:02.138990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e159b-fb2e-4b6d-bbf2-bc5cf9b9e293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:02.140630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e159b-fb2e-4b6d-bbf2-bc5cf9b9e293 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:02.259443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdda825-cdb5-4b0b-b0eb-8ac4168280c7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:02.289377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 40/86] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:02.291171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e159b-fb2e-4b6d-bbf2-bc5cf9b9e293 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:02.317051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722fe9d-3261-4cb0-a82b-4dc99f385c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:02.317776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722fe9d-3261-4cb0-a82b-4dc99f385c80 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:02.318080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722fe9d-3261-4cb0-a82b-4dc99f385c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:02.318356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722fe9d-3261-4cb0-a82b-4dc99f385c80 None admin] PATH_INFO: `/v3/users/e22fbb4f1849455e917f7324ffd888a3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:02.319625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722fe9d-3261-4cb0-a82b-4dc99f385c80 None admin] RBAC: Authorizing `identity:get_user(user_id=e22fbb4f1849455e917f7324ffd888a3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:02.321731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722fe9d-3261-4cb0-a82b-4dc99f385c80 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:02.323700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 47/87] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:02 2022] GET /identity/v3/users/e22fbb4f1849455e917f7324ffd888a3 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:02.324459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722fe9d-3261-4cb0-a82b-4dc99f385c80 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:02.386022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:02.388700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:02.447738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e159b-fb2e-4b6d-bbf2-bc5cf9b9e293 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:02.470272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722fe9d-3261-4cb0-a82b-4dc99f385c80 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:02.498149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a5137-7704-4dfd-a79a-8a2d0f3ad517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:02.498609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a5137-7704-4dfd-a79a-8a2d0f3ad517 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:02.499989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a5137-7704-4dfd-a79a-8a2d0f3ad517 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:02.499989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a5137-7704-4dfd-a79a-8a2d0f3ad517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:02.516519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8a94e-83eb-46a0-a500-11a570c6dd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:02.517556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8a94e-83eb-46a0-a500-11a570c6dd53 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:02.518068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8a94e-83eb-46a0-a500-11a570c6dd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:02.519654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8a94e-83eb-46a0-a500-11a570c6dd53 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:02.528435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8a94e-83eb-46a0-a500-11a570c6dd53 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:02.530202 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8a94e-83eb-46a0-a500-11a570c6dd53 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:02.533098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 48/88] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:02 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:02.533734 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8a94e-83eb-46a0-a500-11a570c6dd53 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:02.599222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a5137-7704-4dfd-a79a-8a2d0f3ad517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:02.601162 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a5137-7704-4dfd-a79a-8a2d0f3ad517 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:02.674228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:02.679872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8a94e-83eb-46a0-a500-11a570c6dd53 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:02.694067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:02.720185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0038f-0dd7-4a02-9584-84283c64b891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:02.721143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0038f-0dd7-4a02-9584-84283c64b891 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:02.721496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0038f-0dd7-4a02-9584-84283c64b891 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:02.721834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0038f-0dd7-4a02-9584-84283c64b891 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:02.723463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0038f-0dd7-4a02-9584-84283c64b891 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:02.725470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0038f-0dd7-4a02-9584-84283c64b891 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:02.727103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 49/89] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:02 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:02.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0038f-0dd7-4a02-9584-84283c64b891 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:02.730028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 41/90] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:02.734161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a5137-7704-4dfd-a79a-8a2d0f3ad517 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:02.843971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0038f-0dd7-4a02-9584-84283c64b891 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:02.861440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a5137-7704-4dfd-a79a-8a2d0f3ad517 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:02.886550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a30341-d67d-4f77-bebb-f0f47422b97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:02.887945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30341-d67d-4f77-bebb-f0f47422b97d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:02.888401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30341-d67d-4f77-bebb-f0f47422b97d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:02.888851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30341-d67d-4f77-bebb-f0f47422b97d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:02.906893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8536b9-1c64-4362-9122-c2074a6edbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:02.907592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8536b9-1c64-4362-9122-c2074a6edbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:02.907946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8536b9-1c64-4362-9122-c2074a6edbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:02.908336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8536b9-1c64-4362-9122-c2074a6edbe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:02.909105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8536b9-1c64-4362-9122-c2074a6edbe3 None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea, user.id=5d32a6d718b64e35aca00f733d966fd9) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:02.909560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8536b9-1c64-4362-9122-c2074a6edbe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:02.911252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8536b9-1c64-4362-9122-c2074a6edbe3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:02.926061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 42/91] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:02 2022] GET /identity/v3/role_assignments?user.id=5d32a6d718b64e35aca00f733d966fd9&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:02.927303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8536b9-1c64-4362-9122-c2074a6edbe3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:02.952927 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:02.983362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a30341-d67d-4f77-bebb-f0f47422b97d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:02.985130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:02.987101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a30341-d67d-4f77-bebb-f0f47422b97d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:03.122489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 50/92] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:03.123839 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a30341-d67d-4f77-bebb-f0f47422b97d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:03.157957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8536b9-1c64-4362-9122-c2074a6edbe3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:03.231169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:03.231333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9589fe-78f0-431a-b19d-837f6f21fd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:03.232233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9589fe-78f0-431a-b19d-837f6f21fd25 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:03.232355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9589fe-78f0-431a-b19d-837f6f21fd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:03.232898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9589fe-78f0-431a-b19d-837f6f21fd25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:03.233472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9589fe-78f0-431a-b19d-837f6f21fd25 None admin] RBAC: Adding query filter params (role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea, user.id=e22fbb4f1849455e917f7324ffd888a3) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:03.233788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9589fe-78f0-431a-b19d-837f6f21fd25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:03.235509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9589fe-78f0-431a-b19d-837f6f21fd25 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:03.245415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 43/93] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:02 2022] GET /identity/v3/role_assignments?user.id=e22fbb4f1849455e917f7324ffd888a3&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=799b2f3554434281bf10b2602fbbc09a => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:03.246396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9589fe-78f0-431a-b19d-837f6f21fd25 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:03.266290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a30341-d67d-4f77-bebb-f0f47422b97d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:03.320561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59b68d-753d-4c19-8507-7d1421a5d0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:03.322191 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59b68d-753d-4c19-8507-7d1421a5d0cf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:03.322721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59b68d-753d-4c19-8507-7d1421a5d0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:03.323196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59b68d-753d-4c19-8507-7d1421a5d0cf None admin] PATH_INFO: `/v3/projects/bd29757566ad433d88cb11652ca31b8a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:03.324885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59b68d-753d-4c19-8507-7d1421a5d0cf None admin] RBAC: Authorizing `identity:get_project(project_id=bd29757566ad433d88cb11652ca31b8a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:03.327528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59b68d-753d-4c19-8507-7d1421a5d0cf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:03.330502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 51/94] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:03 2022] GET /identity/v3/projects/bd29757566ad433d88cb11652ca31b8a => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:03.336628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59b68d-753d-4c19-8507-7d1421a5d0cf None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:03.421648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:03.432688 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9589fe-78f0-431a-b19d-837f6f21fd25 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:03.504055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4c408-a194-445d-a537-6003a603eea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:03.504633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4c408-a194-445d-a537-6003a603eea1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:03.504927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4c408-a194-445d-a537-6003a603eea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:03.505193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4c408-a194-445d-a537-6003a603eea1 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:03.508431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4c408-a194-445d-a537-6003a603eea1 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:03.509968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4c408-a194-445d-a537-6003a603eea1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:03.511882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 44/95] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:03 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:03.519339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4c408-a194-445d-a537-6003a603eea1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:03.610004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59b68d-753d-4c19-8507-7d1421a5d0cf None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:03.672915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b288d-9ef8-4c2c-b00c-94a52f877a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:03.673553 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b288d-9ef8-4c2c-b00c-94a52f877a6f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:03.673839 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b288d-9ef8-4c2c-b00c-94a52f877a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:03.674139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b288d-9ef8-4c2c-b00c-94a52f877a6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:03.675093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b288d-9ef8-4c2c-b00c-94a52f877a6f None admin] RBAC: Adding query filter params (role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=bd29757566ad433d88cb11652ca31b8a, user.id=e22fbb4f1849455e917f7324ffd888a3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:03.675477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b288d-9ef8-4c2c-b00c-94a52f877a6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:03.677053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b288d-9ef8-4c2c-b00c-94a52f877a6f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:03.686991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4c408-a194-445d-a537-6003a603eea1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:03.692264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 52/96] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:03 2022] GET /identity/v3/role_assignments?user.id=e22fbb4f1849455e917f7324ffd888a3&scope.project.id=bd29757566ad433d88cb11652ca31b8a&role.id=799b2f3554434281bf10b2602fbbc09a => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:03.743332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eede30c-771d-492e-8cef-21dfcd0d9895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:03.743971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eede30c-771d-492e-8cef-21dfcd0d9895 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:03.744253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eede30c-771d-492e-8cef-21dfcd0d9895 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:03.744600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eede30c-771d-492e-8cef-21dfcd0d9895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:03.745376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eede30c-771d-492e-8cef-21dfcd0d9895 None admin] RBAC: Adding query filter params (role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea, user.id=e22fbb4f1849455e917f7324ffd888a3) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:03.745745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eede30c-771d-492e-8cef-21dfcd0d9895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:03.747296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eede30c-771d-492e-8cef-21dfcd0d9895 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:03.754927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 45/97] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:03 2022] GET /identity/v3/role_assignments?user.id=e22fbb4f1849455e917f7324ffd888a3&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:04.743049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cc661-d30f-4673-b79c-9cba5cd78251 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:04.743548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cc661-d30f-4673-b79c-9cba5cd78251 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:04.743669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cc661-d30f-4673-b79c-9cba5cd78251 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:04.745649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 53/98] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:04.752463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b3592-f2b7-4a64-8e5d-0efda245f46a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:04.754001 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b3592-f2b7-4a64-8e5d-0efda245f46a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:04.754525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b3592-f2b7-4a64-8e5d-0efda245f46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:04.780606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8d7b3592-f2b7-4a64-8e5d-0efda245f46a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:04.782176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8d7b3592-f2b7-4a64-8e5d-0efda245f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:04.801103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:04.803752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d7c6b-8b2c-45c9-97e9-8c16b24ed65e None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:04.804319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d7c6b-8b2c-45c9-97e9-8c16b24ed65e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:04.804813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d7c6b-8b2c-45c9-97e9-8c16b24ed65e None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:04.806610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 54/99] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:04:04.812730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b82fa-08b7-44ea-a52f-0f98449058d9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:04.813302 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b82fa-08b7-44ea-a52f-0f98449058d9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:04.813743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b82fa-08b7-44ea-a52f-0f98449058d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:04.855168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d27b82fa-08b7-44ea-a52f-0f98449058d9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:04.858199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d27b82fa-08b7-44ea-a52f-0f98449058d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:04.870616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:04.947071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 46/100] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:04.992406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 55/101] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:05.465405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b3592-f2b7-4a64-8e5d-0efda245f46a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:05.564806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b82fa-08b7-44ea-a52f-0f98449058d9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:05.612131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b3592-f2b7-4a64-8e5d-0efda245f46a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:05.719525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989cfb8-cb73-458c-8fb2-95098d3f60de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:05.720203 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b82fa-08b7-44ea-a52f-0f98449058d9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:05.720769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989cfb8-cb73-458c-8fb2-95098d3f60de None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:05.721278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989cfb8-cb73-458c-8fb2-95098d3f60de None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:05.721777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989cfb8-cb73-458c-8fb2-95098d3f60de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:05.749332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989cfb8-cb73-458c-8fb2-95098d3f60de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-7989cfb8-cb73-458c-8fb2-95098d3f60de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:05.755346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:05.756632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:05.757389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:05.768055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 47/102] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:05.773974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989cfb8-cb73-458c-8fb2-95098d3f60de None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:05.786649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aee461-78be-4b98-b32b-0c3e58dec374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:05.787324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aee461-78be-4b98-b32b-0c3e58dec374 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:05.787614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aee461-78be-4b98-b32b-0c3e58dec374 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:05.787898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aee461-78be-4b98-b32b-0c3e58dec374 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:05.789110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aee461-78be-4b98-b32b-0c3e58dec374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:05.793920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aee461-78be-4b98-b32b-0c3e58dec374 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:05.793920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 56/103] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:05.798496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:05.800637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aee461-78be-4b98-b32b-0c3e58dec374 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:05.838309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:05.989044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989cfb8-cb73-458c-8fb2-95098d3f60de None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:06.035924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aee461-78be-4b98-b32b-0c3e58dec374 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:06.086683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2464c-c188-45b0-b44e-e34ff3b89928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:06.087356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2464c-c188-45b0-b44e-e34ff3b89928 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.087601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2464c-c188-45b0-b44e-e34ff3b89928 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.088034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2464c-c188-45b0-b44e-e34ff3b89928 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.088869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2464c-c188-45b0-b44e-e34ff3b89928 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:06.089312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2464c-c188-45b0-b44e-e34ff3b89928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:06.090919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2464c-c188-45b0-b44e-e34ff3b89928 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:06.105194 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 48/104] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:06.113921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2464c-c188-45b0-b44e-e34ff3b89928 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:06.116922 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6535baa-511a-427f-a145-6e1d27999302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:06.121902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6535baa-511a-427f-a145-6e1d27999302 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.121902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6535baa-511a-427f-a145-6e1d27999302 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.121902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6535baa-511a-427f-a145-6e1d27999302 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.121902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6535baa-511a-427f-a145-6e1d27999302 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:06.121902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6535baa-511a-427f-a145-6e1d27999302 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:06.224748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:06.261248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2464c-c188-45b0-b44e-e34ff3b89928 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:06.270098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 57/105] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:04:05 2022] POST /identity/v3/projects => generated 302 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:06.271717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b4b17-a704-4578-ba88-46db1e9513ef None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.272201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b4b17-a704-4578-ba88-46db1e9513ef None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.272664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b4b17-a704-4578-ba88-46db1e9513ef None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.274584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 58/106] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:06.281214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d48d5-e442-4a8a-af97-76e75b41908e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.281789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d48d5-e442-4a8a-af97-76e75b41908e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.282320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d48d5-e442-4a8a-af97-76e75b41908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.303894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a65d-a9b5-4786-b6be-a4e26edaea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:06.304560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a65d-a9b5-4786-b6be-a4e26edaea3e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.304842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a65d-a9b5-4786-b6be-a4e26edaea3e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.305693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a65d-a9b5-4786-b6be-a4e26edaea3e None admin] PATH_INFO: `/v3/users/725fd21c3867436692a2e10ed4770378` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.323326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a65d-a9b5-4786-b6be-a4e26edaea3e None admin] RBAC: Authorizing `identity:get_user(user_id=725fd21c3867436692a2e10ed4770378)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:06.486464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a65d-a9b5-4786-b6be-a4e26edaea3e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:06.486464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 49/107] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:06 2022] GET /identity/v3/users/725fd21c3867436692a2e10ed4770378 => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:06.486464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a65d-a9b5-4786-b6be-a4e26edaea3e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:06.487778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c51d48d5-e442-4a8a-af97-76e75b41908e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:06.487778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c51d48d5-e442-4a8a-af97-76e75b41908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:06.495659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:06.530919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 59/108] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:06.532004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbea20-6e9f-42e2-95be-6b1fdbead513 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.532369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbea20-6e9f-42e2-95be-6b1fdbead513 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.532720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbea20-6e9f-42e2-95be-6b1fdbead513 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.535012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 60/109] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:06.540653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69953a0c-98c9-40a8-9864-e2e417e98c24 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.540938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69953a0c-98c9-40a8-9864-e2e417e98c24 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.541522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69953a0c-98c9-40a8-9864-e2e417e98c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.578464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-69953a0c-98c9-40a8-9864-e2e417e98c24 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:06.579712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-69953a0c-98c9-40a8-9864-e2e417e98c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:06.609028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a65d-a9b5-4786-b6be-a4e26edaea3e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:06.665404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:06.685533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 61/110] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:06.686886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d17666-6376-46ab-8072-183488cbe397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:06.688164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d17666-6376-46ab-8072-183488cbe397 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.688833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d17666-6376-46ab-8072-183488cbe397 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.689279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d17666-6376-46ab-8072-183488cbe397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.780762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369051b9-8982-4609-b077-1544a7bf8375 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.781701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369051b9-8982-4609-b077-1544a7bf8375 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.782453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369051b9-8982-4609-b077-1544a7bf8375 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.784211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d17666-6376-46ab-8072-183488cbe397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:06.787470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d17666-6376-46ab-8072-183488cbe397 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:06.788353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 62/111] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:06.797459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e65fd-8545-4b53-a1f9-fe0cee0298cc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.798347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e65fd-8545-4b53-a1f9-fe0cee0298cc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.799127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e65fd-8545-4b53-a1f9-fe0cee0298cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.833368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1f0e65fd-8545-4b53-a1f9-fe0cee0298cc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:06.835891 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1f0e65fd-8545-4b53-a1f9-fe0cee0298cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:06.958827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 50/112] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:06.960491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aba0a-ece9-4a8f-bbb7-7120e84a142a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.960925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aba0a-ece9-4a8f-bbb7-7120e84a142a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.961394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aba0a-ece9-4a8f-bbb7-7120e84a142a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:06.962899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 51/113] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:06.964906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568aba0a-ece9-4a8f-bbb7-7120e84a142a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:06.994051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 63/114] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:06.998274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e5e02-34cb-45c3-b603-a95f3a2a011f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:06.998774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e5e02-34cb-45c3-b603-a95f3a2a011f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:06.999278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e5e02-34cb-45c3-b603-a95f3a2a011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:07.014704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:07.051675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4f8e5e02-34cb-45c3-b603-a95f3a2a011f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:07.054062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4f8e5e02-34cb-45c3-b603-a95f3a2a011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:07.103563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568aba0a-ece9-4a8f-bbb7-7120e84a142a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:07.146039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8474fe-972b-401a-b93c-ec10685d35f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:07.147066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8474fe-972b-401a-b93c-ec10685d35f6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:07.147551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8474fe-972b-401a-b93c-ec10685d35f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:07.148185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8474fe-972b-401a-b93c-ec10685d35f6 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:07.150054 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8474fe-972b-401a-b93c-ec10685d35f6 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:07.151764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:07.152280 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8474fe-972b-401a-b93c-ec10685d35f6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:07.154179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 52/115] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:06 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:07.167340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8474fe-972b-401a-b93c-ec10685d35f6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:07.188987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 64/116] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:07.190978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e5e02-34cb-45c3-b603-a95f3a2a011f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:07.327819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:07.332748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8474fe-972b-401a-b93c-ec10685d35f6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:07.392357 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e5e02-34cb-45c3-b603-a95f3a2a011f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:07.396651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb2ca7-50f2-4f6f-896c-515cf5300841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:07.397420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb2ca7-50f2-4f6f-896c-515cf5300841 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:07.397808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb2ca7-50f2-4f6f-896c-515cf5300841 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:07.398215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb2ca7-50f2-4f6f-896c-515cf5300841 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:07.410497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb2ca7-50f2-4f6f-896c-515cf5300841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c5fb2ca7-50f2-4f6f-896c-515cf5300841 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:07.415523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:07.416631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:07.417604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:07.419329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 53/117] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:07.420251 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb2ca7-50f2-4f6f-896c-515cf5300841 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:07.444786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82dc1d-3a9f-46ac-89ba-e929c1ae94ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:07.445737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82dc1d-3a9f-46ac-89ba-e929c1ae94ae None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:07.446061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82dc1d-3a9f-46ac-89ba-e929c1ae94ae None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:07.446339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82dc1d-3a9f-46ac-89ba-e929c1ae94ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:07.460135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82dc1d-3a9f-46ac-89ba-e929c1ae94ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7b82dc1d-3a9f-46ac-89ba-e929c1ae94ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:07.467112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:07.468025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:07.468872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 65/118] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:04:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:07.469520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82dc1d-3a9f-46ac-89ba-e929c1ae94ae None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:07.528837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:07.562843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb2ca7-50f2-4f6f-896c-515cf5300841 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:07.598547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82dc1d-3a9f-46ac-89ba-e929c1ae94ae None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:07.601891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde289-d936-4869-8b74-1d96a1e0ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:07.602506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde289-d936-4869-8b74-1d96a1e0ba9b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:07.602785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde289-d936-4869-8b74-1d96a1e0ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:07.603060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde289-d936-4869-8b74-1d96a1e0ba9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:07.603877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccde289-d936-4869-8b74-1d96a1e0ba9b None admin] RBAC: Adding query filter params (role.id=0f2f4356fa294a7db98c5636fe2d390d, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea, user.id=725fd21c3867436692a2e10ed4770378) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:07.604102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccde289-d936-4869-8b74-1d96a1e0ba9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:07.605566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccde289-d936-4869-8b74-1d96a1e0ba9b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:07.621918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 54/119] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:07 2022] GET /identity/v3/role_assignments?user.id=725fd21c3867436692a2e10ed4770378&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=0f2f4356fa294a7db98c5636fe2d390d => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:07.621918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde289-d936-4869-8b74-1d96a1e0ba9b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:07.655605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:07.658558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1d21e-4696-4190-844f-91a42923f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:07.659287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1d21e-4696-4190-844f-91a42923f46a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:07.659572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1d21e-4696-4190-844f-91a42923f46a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:07.660027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1d21e-4696-4190-844f-91a42923f46a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:07.676237 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1d21e-4696-4190-844f-91a42923f46a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-98c1d21e-4696-4190-844f-91a42923f46a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:07.683095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:07.684038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:07.685143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 66/120] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:07.685984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1d21e-4696-4190-844f-91a42923f46a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:07.747830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde289-d936-4869-8b74-1d96a1e0ba9b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:07.798769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc653fc6-7048-4e3a-90f4-7af21d0c4ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:07.800498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc653fc6-7048-4e3a-90f4-7af21d0c4ebf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:07.800971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc653fc6-7048-4e3a-90f4-7af21d0c4ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:07.801443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc653fc6-7048-4e3a-90f4-7af21d0c4ebf None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:07.802361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc653fc6-7048-4e3a-90f4-7af21d0c4ebf None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:07.802913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc653fc6-7048-4e3a-90f4-7af21d0c4ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:07.804783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc653fc6-7048-4e3a-90f4-7af21d0c4ebf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:07.816348 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 55/121] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:07.817686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc653fc6-7048-4e3a-90f4-7af21d0c4ebf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:07.828552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:07.833661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1d21e-4696-4190-844f-91a42923f46a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:07.879277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139701b-5e27-4476-a7ed-90add073b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:07.879825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139701b-5e27-4476-a7ed-90add073b0ce None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:07.880107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139701b-5e27-4476-a7ed-90add073b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:07.880386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139701b-5e27-4476-a7ed-90add073b0ce None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:07.881397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139701b-5e27-4476-a7ed-90add073b0ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:07.881714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139701b-5e27-4476-a7ed-90add073b0ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:07.883470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139701b-5e27-4476-a7ed-90add073b0ce None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:07.896551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 67/122] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:07.898247 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139701b-5e27-4476-a7ed-90add073b0ce None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:07.924450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:07.944681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc653fc6-7048-4e3a-90f4-7af21d0c4ebf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:07.988610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222f435-7da1-4140-9bc8-57703358c3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:07.989777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222f435-7da1-4140-9bc8-57703358c3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:07.990367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222f435-7da1-4140-9bc8-57703358c3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:07.990841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222f435-7da1-4140-9bc8-57703358c3a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.005217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222f435-7da1-4140-9bc8-57703358c3a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a222f435-7da1-4140-9bc8-57703358c3a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:08.011097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:08.011972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:08.013099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:08.015776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 56/123] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:08.016828 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222f435-7da1-4140-9bc8-57703358c3a5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:08.058037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139701b-5e27-4476-a7ed-90add073b0ce None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:08.111557 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b554b-c058-432e-9f4a-40361194a607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:08.112183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b554b-c058-432e-9f4a-40361194a607 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:08.112468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b554b-c058-432e-9f4a-40361194a607 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:08.112744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b554b-c058-432e-9f4a-40361194a607 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.113432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b554b-c058-432e-9f4a-40361194a607 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:08.114278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b554b-c058-432e-9f4a-40361194a607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:08.116182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b554b-c058-432e-9f4a-40361194a607 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:08.127922 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 68/124] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:08.132971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b554b-c058-432e-9f4a-40361194a607 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:08.146970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222f435-7da1-4140-9bc8-57703358c3a5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:08.164465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:08.191152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aa2b0-8d5c-4df1-aae4-a576e8e67e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:08.191962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aa2b0-8d5c-4df1-aae4-a576e8e67e09 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:08.192251 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aa2b0-8d5c-4df1-aae4-a576e8e67e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:08.192533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aa2b0-8d5c-4df1-aae4-a576e8e67e09 None admin] PATH_INFO: `/v3/users/e22fbb4f1849455e917f7324ffd888a3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.193826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aa2b0-8d5c-4df1-aae4-a576e8e67e09 None admin] RBAC: Authorizing `identity:get_user(user_id=e22fbb4f1849455e917f7324ffd888a3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:08.195694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aa2b0-8d5c-4df1-aae4-a576e8e67e09 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:08.197907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 57/125] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:08 2022] GET /identity/v3/users/e22fbb4f1849455e917f7324ffd888a3 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:08.198794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aa2b0-8d5c-4df1-aae4-a576e8e67e09 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:08.219481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:08.277087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b554b-c058-432e-9f4a-40361194a607 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:08.333002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ace5f-63db-49f4-bc1c-1a9a49fe8f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:08.333460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ace5f-63db-49f4-bc1c-1a9a49fe8f5d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:08.333754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ace5f-63db-49f4-bc1c-1a9a49fe8f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:08.334070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ace5f-63db-49f4-bc1c-1a9a49fe8f5d None admin] PATH_INFO: `/v3/users/5d32a6d718b64e35aca00f733d966fd9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.335271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ace5f-63db-49f4-bc1c-1a9a49fe8f5d None admin] RBAC: Authorizing `identity:get_user(user_id=5d32a6d718b64e35aca00f733d966fd9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:08.336817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ace5f-63db-49f4-bc1c-1a9a49fe8f5d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:08.337396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aa2b0-8d5c-4df1-aae4-a576e8e67e09 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:08.338927 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 69/126] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:08 2022] GET /identity/v3/users/5d32a6d718b64e35aca00f733d966fd9 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:08.340061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ace5f-63db-49f4-bc1c-1a9a49fe8f5d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:08.394519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d166b9-5857-44d8-ba55-038ad38401d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:08.395618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d166b9-5857-44d8-ba55-038ad38401d0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:08.396119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d166b9-5857-44d8-ba55-038ad38401d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:08.396980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d166b9-5857-44d8-ba55-038ad38401d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.398594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d166b9-5857-44d8-ba55-038ad38401d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:08.399686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d166b9-5857-44d8-ba55-038ad38401d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:08.402909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d166b9-5857-44d8-ba55-038ad38401d0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:08.415262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 58/127] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:08.416536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d166b9-5857-44d8-ba55-038ad38401d0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:08.485598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:08.510324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ace5f-63db-49f4-bc1c-1a9a49fe8f5d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:08.545431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:08.575128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53890aa8-4089-4a26-a0a0-7078468d13f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:08.575895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890aa8-4089-4a26-a0a0-7078468d13f2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:08.576299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890aa8-4089-4a26-a0a0-7078468d13f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:08.576683 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890aa8-4089-4a26-a0a0-7078468d13f2 None admin] PATH_INFO: `/v3/users/e22fbb4f1849455e917f7324ffd888a3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.578040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53890aa8-4089-4a26-a0a0-7078468d13f2 None admin] RBAC: Authorizing `identity:get_user(user_id=e22fbb4f1849455e917f7324ffd888a3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:08.579743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53890aa8-4089-4a26-a0a0-7078468d13f2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:08.582617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 70/128] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:08 2022] GET /identity/v3/users/e22fbb4f1849455e917f7324ffd888a3 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:08.587697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53890aa8-4089-4a26-a0a0-7078468d13f2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:08.613282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d166b9-5857-44d8-ba55-038ad38401d0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:08.694472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9ddb9-dd30-430f-ae63-2ef13bf294ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:08.696015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9ddb9-dd30-430f-ae63-2ef13bf294ce None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:08.696493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9ddb9-dd30-430f-ae63-2ef13bf294ce None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:08.697023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9ddb9-dd30-430f-ae63-2ef13bf294ce None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.699940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9ddb9-dd30-430f-ae63-2ef13bf294ce None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:08.702937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9ddb9-dd30-430f-ae63-2ef13bf294ce None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:08.707348 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 59/129] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:08 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:08.708547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9ddb9-dd30-430f-ae63-2ef13bf294ce None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:08.727138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53890aa8-4089-4a26-a0a0-7078468d13f2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:08.767647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f3f39-279a-4e4e-b747-bf868afbe71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:08.768894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f3f39-279a-4e4e-b747-bf868afbe71c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:08.769274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f3f39-279a-4e4e-b747-bf868afbe71c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:08.769628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f3f39-279a-4e4e-b747-bf868afbe71c None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.771167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f3f39-279a-4e4e-b747-bf868afbe71c None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:08.772878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f3f39-279a-4e4e-b747-bf868afbe71c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:08.775196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 71/130] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:08 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:08.776180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f3f39-279a-4e4e-b747-bf868afbe71c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:08.809066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:08.845670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9ddb9-dd30-430f-ae63-2ef13bf294ce None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:08.898772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f3f39-279a-4e4e-b747-bf868afbe71c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:08.902858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fb624-b7cc-4cd9-b662-150ef9964e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:08.903749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fb624-b7cc-4cd9-b662-150ef9964e18 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:08.904209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fb624-b7cc-4cd9-b662-150ef9964e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:08.904724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fb624-b7cc-4cd9-b662-150ef9964e18 None admin] PATH_INFO: `/v3/users/e22fbb4f1849455e917f7324ffd888a3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.906318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fb624-b7cc-4cd9-b662-150ef9964e18 None admin] RBAC: Authorizing `identity:get_user(user_id=e22fbb4f1849455e917f7324ffd888a3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:08.908324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fb624-b7cc-4cd9-b662-150ef9964e18 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:08.910803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 60/131] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:08 2022] GET /identity/v3/users/e22fbb4f1849455e917f7324ffd888a3 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:08.911821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fb624-b7cc-4cd9-b662-150ef9964e18 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:08.928045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:08.947372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe2257-9a3d-4639-8d21-3432048b90ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:08.947939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe2257-9a3d-4639-8d21-3432048b90ee None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:08.948226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe2257-9a3d-4639-8d21-3432048b90ee None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:08.948512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe2257-9a3d-4639-8d21-3432048b90ee None admin] PATH_INFO: `/v3/projects/bd29757566ad433d88cb11652ca31b8a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:08.950738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe2257-9a3d-4639-8d21-3432048b90ee None admin] RBAC: Authorizing `identity:get_project(project_id=bd29757566ad433d88cb11652ca31b8a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:08.952118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe2257-9a3d-4639-8d21-3432048b90ee None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:08.953965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 72/132] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:08 2022] GET /identity/v3/projects/bd29757566ad433d88cb11652ca31b8a => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:08.958911 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe2257-9a3d-4639-8d21-3432048b90ee None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:09.056942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fb624-b7cc-4cd9-b662-150ef9964e18 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:09.083236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:09.092148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe2257-9a3d-4639-8d21-3432048b90ee None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:09.113248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c328f-8820-4818-8d07-6232858f76c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:09.113957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c328f-8820-4818-8d07-6232858f76c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:09.114245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c328f-8820-4818-8d07-6232858f76c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:09.114527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c328f-8820-4818-8d07-6232858f76c7 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/users/e22fbb4f1849455e917f7324ffd888a3/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:09.116538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c328f-8820-4818-8d07-6232858f76c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a747fd9ec2214dcc9ddec8091b57baea, user_id=e22fbb4f1849455e917f7324ffd888a3, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:09.118167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c328f-8820-4818-8d07-6232858f76c7 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:09.145246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83777e-da06-4507-9f5c-7efc5bd69ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:09.146108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83777e-da06-4507-9f5c-7efc5bd69ec9 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:09.146388 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83777e-da06-4507-9f5c-7efc5bd69ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:09.146665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83777e-da06-4507-9f5c-7efc5bd69ec9 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/users/5d32a6d718b64e35aca00f733d966fd9/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:09.148675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83777e-da06-4507-9f5c-7efc5bd69ec9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a747fd9ec2214dcc9ddec8091b57baea, user_id=5d32a6d718b64e35aca00f733d966fd9, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:09.150599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83777e-da06-4507-9f5c-7efc5bd69ec9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:09.266203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 61/133] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:04:08 2022] PUT /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/users/e22fbb4f1849455e917f7324ffd888a3/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:09.266762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c328f-8820-4818-8d07-6232858f76c7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:09.297416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 73/134] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:04:08 2022] PUT /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/users/5d32a6d718b64e35aca00f733d966fd9/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:09.297416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936576a7-e564-4d72-a651-da4720e55dad None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:09.297416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936576a7-e564-4d72-a651-da4720e55dad None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:09.297416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936576a7-e564-4d72-a651-da4720e55dad None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:09.297416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 74/135] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:09.297416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936576a7-e564-4d72-a651-da4720e55dad None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:09.385365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:09.428607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c328f-8820-4818-8d07-6232858f76c7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:09.509710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9e943-b12c-4142-9ff1-5a7c25e3d3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:09.510380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9e943-b12c-4142-9ff1-5a7c25e3d3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:09.510740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9e943-b12c-4142-9ff1-5a7c25e3d3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:09.511134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9e943-b12c-4142-9ff1-5a7c25e3d3a4 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:09.512399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9e943-b12c-4142-9ff1-5a7c25e3d3a4 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:09.514023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9e943-b12c-4142-9ff1-5a7c25e3d3a4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:09.517844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 62/136] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:09 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:09.519936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a928e-4659-4572-9090-70b4157ea5f5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:09.520282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a928e-4659-4572-9090-70b4157ea5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:09.520639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a928e-4659-4572-9090-70b4157ea5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:09.542085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-110a928e-4659-4572-9090-70b4157ea5f5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:09.543840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-110a928e-4659-4572-9090-70b4157ea5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:09.568002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936576a7-e564-4d72-a651-da4720e55dad None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:09.611485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2ac7c-5903-4e5d-b799-81525af720b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:09.613213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2ac7c-5903-4e5d-b799-81525af720b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:09.613672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2ac7c-5903-4e5d-b799-81525af720b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:09.614181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2ac7c-5903-4e5d-b799-81525af720b8 None admin] PATH_INFO: `/v3/projects/bd29757566ad433d88cb11652ca31b8a/users/e22fbb4f1849455e917f7324ffd888a3/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:09.616766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2ac7c-5903-4e5d-b799-81525af720b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd29757566ad433d88cb11652ca31b8a, user_id=e22fbb4f1849455e917f7324ffd888a3, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:09.618436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2ac7c-5903-4e5d-b799-81525af720b8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:09.644622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 63/137] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:09.645924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a928e-4659-4572-9090-70b4157ea5f5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:09.708566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 75/138] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:04:09 2022] PUT /identity/v3/projects/bd29757566ad433d88cb11652ca31b8a/users/e22fbb4f1849455e917f7324ffd888a3/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:09.716159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38247676-6839-4afa-8dd3-55176d97258f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:09.716538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38247676-6839-4afa-8dd3-55176d97258f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:09.716768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38247676-6839-4afa-8dd3-55176d97258f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:09.718257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 76/139] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:09.756150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c2428-610d-4d33-9bcb-b0a0b6c457ec None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:09.756432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c2428-610d-4d33-9bcb-b0a0b6c457ec None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:09.756726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c2428-610d-4d33-9bcb-b0a0b6c457ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:09.761103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:09.788326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ed3c2428-610d-4d33-9bcb-b0a0b6c457ec None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:09.789630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ed3c2428-610d-4d33-9bcb-b0a0b6c457ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:09.814269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:09.815881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a928e-4659-4572-9090-70b4157ea5f5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:09.873985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f34560-9105-4527-ae2b-0e4f7eb46758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:09.885370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f34560-9105-4527-ae2b-0e4f7eb46758 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:09.885370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f34560-9105-4527-ae2b-0e4f7eb46758 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:09.885370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f34560-9105-4527-ae2b-0e4f7eb46758 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/users/e22fbb4f1849455e917f7324ffd888a3/roles/e08fcbbf38b344f48f71ed9ceb314293` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:09.885662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 77/140] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:09.885747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f34560-9105-4527-ae2b-0e4f7eb46758 None admin] RBAC: Authorizing `identity:create_grant(project_id=a747fd9ec2214dcc9ddec8091b57baea, user_id=e22fbb4f1849455e917f7324ffd888a3, role_id=e08fcbbf38b344f48f71ed9ceb314293)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:09.885747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f34560-9105-4527-ae2b-0e4f7eb46758 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:10.071016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 64/141] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:04:09 2022] PUT /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/users/e22fbb4f1849455e917f7324ffd888a3/roles/e08fcbbf38b344f48f71ed9ceb314293 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:10.093685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c2428-610d-4d33-9bcb-b0a0b6c457ec None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:10.253006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c2428-610d-4d33-9bcb-b0a0b6c457ec None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:10.267003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f34560-9105-4527-ae2b-0e4f7eb46758 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:10.352034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75b9b3-0470-4964-9f23-9d6a41249c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:10.352761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75b9b3-0470-4964-9f23-9d6a41249c40 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:10.353049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75b9b3-0470-4964-9f23-9d6a41249c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:10.353329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75b9b3-0470-4964-9f23-9d6a41249c40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:10.356592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75b9b3-0470-4964-9f23-9d6a41249c40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:10.358105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75b9b3-0470-4964-9f23-9d6a41249c40 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:10.359943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 78/142] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:10.366449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75b9b3-0470-4964-9f23-9d6a41249c40 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:10.414354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:10.432961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f34560-9105-4527-ae2b-0e4f7eb46758 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:10.478002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:10.487509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ca866-c302-49ab-a465-2ef75a8ee8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:10.488421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ca866-c302-49ab-a465-2ef75a8ee8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:10.488846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ca866-c302-49ab-a465-2ef75a8ee8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:10.489297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ca866-c302-49ab-a465-2ef75a8ee8d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:10.501935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ca866-c302-49ab-a465-2ef75a8ee8d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e22ca866-c302-49ab-a465-2ef75a8ee8d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:10.507441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:10.508400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:10.509466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:10.512994 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 65/143] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:10.526896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ca866-c302-49ab-a465-2ef75a8ee8d5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:10.655796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ca866-c302-49ab-a465-2ef75a8ee8d5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:10.700150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e15fa1-0f75-41cb-9ca2-8be5c39738e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:10.701192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15fa1-0f75-41cb-9ca2-8be5c39738e0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:10.701626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15fa1-0f75-41cb-9ca2-8be5c39738e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:10.702477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15fa1-0f75-41cb-9ca2-8be5c39738e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:10.703465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e15fa1-0f75-41cb-9ca2-8be5c39738e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:10.704322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e15fa1-0f75-41cb-9ca2-8be5c39738e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:10.707070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e15fa1-0f75-41cb-9ca2-8be5c39738e0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:10.720443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 66/144] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:10.728450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e15fa1-0f75-41cb-9ca2-8be5c39738e0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:10.750742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75b9b3-0470-4964-9f23-9d6a41249c40 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:10.792207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff01fb-6b68-42e0-bf76-0897c477a922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:10.793234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff01fb-6b68-42e0-bf76-0897c477a922 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:10.793663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff01fb-6b68-42e0-bf76-0897c477a922 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:10.794144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff01fb-6b68-42e0-bf76-0897c477a922 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:10.794962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff01fb-6b68-42e0-bf76-0897c477a922 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:10.797696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff01fb-6b68-42e0-bf76-0897c477a922 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:10.878517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:10.942244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e15fa1-0f75-41cb-9ca2-8be5c39738e0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:10.974063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 79/145] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:10 2022] POST /identity/v3/users => generated 283 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:10.990039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea924b3-7a41-4d77-a077-6646da4e54f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:10.990706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea924b3-7a41-4d77-a077-6646da4e54f8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:10.990998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea924b3-7a41-4d77-a077-6646da4e54f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:10.991441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea924b3-7a41-4d77-a077-6646da4e54f8 None admin] PATH_INFO: `/v3/users/725fd21c3867436692a2e10ed4770378` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:10.992699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea924b3-7a41-4d77-a077-6646da4e54f8 None admin] RBAC: Authorizing `identity:get_user(user_id=725fd21c3867436692a2e10ed4770378)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:10.994311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea924b3-7a41-4d77-a077-6646da4e54f8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:10.996285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 67/146] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:10 2022] GET /identity/v3/users/725fd21c3867436692a2e10ed4770378 => generated 289 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:11.002163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff01fb-6b68-42e0-bf76-0897c477a922 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:11.123643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:11.164158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff01fb-6b68-42e0-bf76-0897c477a922 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:11.207931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab72fba-4309-4cbc-9dfa-584b54140d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:11.208818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab72fba-4309-4cbc-9dfa-584b54140d52 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:11.209238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab72fba-4309-4cbc-9dfa-584b54140d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:11.209659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab72fba-4309-4cbc-9dfa-584b54140d52 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:11.214674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab72fba-4309-4cbc-9dfa-584b54140d52 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:11.216471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab72fba-4309-4cbc-9dfa-584b54140d52 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:11.218454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 80/147] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:11 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:11.227289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea924b3-7a41-4d77-a077-6646da4e54f8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:11.403935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea924b3-7a41-4d77-a077-6646da4e54f8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:11.425398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:11.443857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851d222-0895-4b16-bc97-b8244a875209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:11.444673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851d222-0895-4b16-bc97-b8244a875209 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:11.444958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851d222-0895-4b16-bc97-b8244a875209 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:11.445234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851d222-0895-4b16-bc97-b8244a875209 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/users/725fd21c3867436692a2e10ed4770378/roles/0f2f4356fa294a7db98c5636fe2d390d` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:11.447279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851d222-0895-4b16-bc97-b8244a875209 None admin] RBAC: Authorizing `identity:create_grant(project_id=a747fd9ec2214dcc9ddec8091b57baea, user_id=725fd21c3867436692a2e10ed4770378, role_id=0f2f4356fa294a7db98c5636fe2d390d)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:11.448880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851d222-0895-4b16-bc97-b8244a875209 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:11.540461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 68/148] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:04:11 2022] PUT /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/users/725fd21c3867436692a2e10ed4770378/roles/0f2f4356fa294a7db98c5636fe2d390d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:11.968925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f089d-48ea-4f77-ae4c-aaab1b8057aa None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:11.969572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f089d-48ea-4f77-ae4c-aaab1b8057aa None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:11.970096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f089d-48ea-4f77-ae4c-aaab1b8057aa None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:11.972278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 81/149] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:11.978080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cdb5a-7be6-451f-971f-f9a203b95afc None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:11.978554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cdb5a-7be6-451f-971f-f9a203b95afc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:11.978631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cdb5a-7be6-451f-971f-f9a203b95afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:12.007102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-455cdb5a-7be6-451f-971f-f9a203b95afc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:12.008353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-455cdb5a-7be6-451f-971f-f9a203b95afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:12.099641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 69/150] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:12.242464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571b75e-c05c-493e-ba4b-c13dee34eb18 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:12.243365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571b75e-c05c-493e-ba4b-c13dee34eb18 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:12.243830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571b75e-c05c-493e-ba4b-c13dee34eb18 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:12.245719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 82/151] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:12.250881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83898dc2-43ee-48a2-9f70-7d2d5a4d7212 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:12.251169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83898dc2-43ee-48a2-9f70-7d2d5a4d7212 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:12.251442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83898dc2-43ee-48a2-9f70-7d2d5a4d7212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:12.286662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-83898dc2-43ee-48a2-9f70-7d2d5a4d7212 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:12.287893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-83898dc2-43ee-48a2-9f70-7d2d5a4d7212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:12.310641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cc92b-cfc2-4d2a-946f-3d66314287ed None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:12.310933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cc92b-cfc2-4d2a-946f-3d66314287ed None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:12.311309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cc92b-cfc2-4d2a-946f-3d66314287ed None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:12.312864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 83/152] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:12.318749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7865aff-30d3-4c74-8000-39de837f98e8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:12.319318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7865aff-30d3-4c74-8000-39de837f98e8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:12.319782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7865aff-30d3-4c74-8000-39de837f98e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:12.352475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a7865aff-30d3-4c74-8000-39de837f98e8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:12.354118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a7865aff-30d3-4c74-8000-39de837f98e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:12.406063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 70/153] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:12.473568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83898dc2-43ee-48a2-9f70-7d2d5a4d7212 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:12.484860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 84/154] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:12.526720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:12.603500 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83898dc2-43ee-48a2-9f70-7d2d5a4d7212 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:12.645269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17e33b-6677-44b2-8c35-f6bb9f2564b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:12.645950 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17e33b-6677-44b2-8c35-f6bb9f2564b4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:12.646233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17e33b-6677-44b2-8c35-f6bb9f2564b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:12.646508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17e33b-6677-44b2-8c35-f6bb9f2564b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:12.657845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17e33b-6677-44b2-8c35-f6bb9f2564b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0d17e33b-6677-44b2-8c35-f6bb9f2564b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:12.667003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:12.667885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:12.668686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 71/155] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:04:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:12.674118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7865aff-30d3-4c74-8000-39de837f98e8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:12.756539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:12.792240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7865aff-30d3-4c74-8000-39de837f98e8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:12.834328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e031d8-4950-4c91-914d-aa9baab4d1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:12.834956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e031d8-4950-4c91-914d-aa9baab4d1cf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:12.835233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e031d8-4950-4c91-914d-aa9baab4d1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:12.835491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e031d8-4950-4c91-914d-aa9baab4d1cf None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:12.836179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e031d8-4950-4c91-914d-aa9baab4d1cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:12.836479 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e031d8-4950-4c91-914d-aa9baab4d1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:12.837902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e031d8-4950-4c91-914d-aa9baab4d1cf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:12.851133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17e33b-6677-44b2-8c35-f6bb9f2564b4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:12.852526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 85/156] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:12.857965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e031d8-4950-4c91-914d-aa9baab4d1cf None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:12.989724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e031d8-4950-4c91-914d-aa9baab4d1cf None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:13.018019 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:13.025922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17e33b-6677-44b2-8c35-f6bb9f2564b4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:13.027325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:13.033234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c00fc0-d565-4e2c-9d7d-fe8fa3e8e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:13.034470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c00fc0-d565-4e2c-9d7d-fe8fa3e8e72e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:13.034730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c00fc0-d565-4e2c-9d7d-fe8fa3e8e72e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:13.034992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c00fc0-d565-4e2c-9d7d-fe8fa3e8e72e None admin] PATH_INFO: `/v3/users/5d32a6d718b64e35aca00f733d966fd9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:13.036173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c00fc0-d565-4e2c-9d7d-fe8fa3e8e72e None admin] RBAC: Authorizing `identity:get_user(user_id=5d32a6d718b64e35aca00f733d966fd9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:13.037651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c00fc0-d565-4e2c-9d7d-fe8fa3e8e72e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:13.039662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 86/157] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:12 2022] GET /identity/v3/users/5d32a6d718b64e35aca00f733d966fd9 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:13.045218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c00fc0-d565-4e2c-9d7d-fe8fa3e8e72e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:13.082532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520b86b-8e2e-4834-82d5-706e91ef4cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:13.084047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520b86b-8e2e-4834-82d5-706e91ef4cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:13.084615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520b86b-8e2e-4834-82d5-706e91ef4cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:13.085104 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520b86b-8e2e-4834-82d5-706e91ef4cc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:13.099637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520b86b-8e2e-4834-82d5-706e91ef4cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4520b86b-8e2e-4834-82d5-706e91ef4cc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:13.105992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:13.106959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:13.107713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:13.110159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 72/158] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:13.116860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520b86b-8e2e-4834-82d5-706e91ef4cc9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:13.264466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520b86b-8e2e-4834-82d5-706e91ef4cc9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:13.304049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c00fc0-d565-4e2c-9d7d-fe8fa3e8e72e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:13.322293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daef12-8849-4168-8328-b015ece30ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:13.322293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daef12-8849-4168-8328-b015ece30ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:13.322293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daef12-8849-4168-8328-b015ece30ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:13.322293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daef12-8849-4168-8328-b015ece30ec1 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:13.322824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daef12-8849-4168-8328-b015ece30ec1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:13.322824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daef12-8849-4168-8328-b015ece30ec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:13.323982 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daef12-8849-4168-8328-b015ece30ec1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:13.349011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 73/159] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:13.353114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daef12-8849-4168-8328-b015ece30ec1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:13.379010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2b5ff-010c-4c4a-9b0a-448748df6bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:13.379643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2b5ff-010c-4c4a-9b0a-448748df6bbe None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:13.379919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2b5ff-010c-4c4a-9b0a-448748df6bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:13.380187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2b5ff-010c-4c4a-9b0a-448748df6bbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:13.398295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:13.532942 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:13.539363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2b5ff-010c-4c4a-9b0a-448748df6bbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:13.540860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2b5ff-010c-4c4a-9b0a-448748df6bbe None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:13.543883 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daef12-8849-4168-8328-b015ece30ec1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:13.653087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bc51a-e454-4859-901b-78219f4116f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:13.654187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bc51a-e454-4859-901b-78219f4116f2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:13.654671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bc51a-e454-4859-901b-78219f4116f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:13.655124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bc51a-e454-4859-901b-78219f4116f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:13.676155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bc51a-e454-4859-901b-78219f4116f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-309bc51a-e454-4859-901b-78219f4116f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:13.690867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:13.691836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:13.692933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:13.700329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 74/160] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:13.704248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fac236-e167-49d1-bd00-f9549d3299fa None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:13.707321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fac236-e167-49d1-bd00-f9549d3299fa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:13.716131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fac236-e167-49d1-bd00-f9549d3299fa None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:13.717809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 75/161] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:13 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:13.718561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fac236-e167-49d1-bd00-f9549d3299fa None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:13.800729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 87/162] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 756 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:04:13.802439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2b5ff-010c-4c4a-9b0a-448748df6bbe None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:13.882211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:13.924870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fac236-e167-49d1-bd00-f9549d3299fa None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:14.012416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef6c9f-1ba3-4dd8-809b-6e5f79f903b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:14.013060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef6c9f-1ba3-4dd8-809b-6e5f79f903b1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:14.013339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef6c9f-1ba3-4dd8-809b-6e5f79f903b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:14.013616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef6c9f-1ba3-4dd8-809b-6e5f79f903b1 None admin] PATH_INFO: `/v3/users/e22fbb4f1849455e917f7324ffd888a3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:14.014941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef6c9f-1ba3-4dd8-809b-6e5f79f903b1 None admin] RBAC: Authorizing `identity:get_user(user_id=e22fbb4f1849455e917f7324ffd888a3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:14.016399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef6c9f-1ba3-4dd8-809b-6e5f79f903b1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:14.018583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 76/163] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:13 2022] GET /identity/v3/users/e22fbb4f1849455e917f7324ffd888a3 => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:14.025283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032de61d-a731-454d-9705-fc73278008da None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:14.025580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032de61d-a731-454d-9705-fc73278008da None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:14.025825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032de61d-a731-454d-9705-fc73278008da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:14.031015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:14.042455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2b5ff-010c-4c4a-9b0a-448748df6bbe None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:14.078357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-032de61d-a731-454d-9705-fc73278008da None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:14.079774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-032de61d-a731-454d-9705-fc73278008da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:14.132634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d6360-71c6-4d34-af8c-f4ff81939d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:14.133718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d6360-71c6-4d34-af8c-f4ff81939d0f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:14.134224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d6360-71c6-4d34-af8c-f4ff81939d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:14.134698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d6360-71c6-4d34-af8c-f4ff81939d0f None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:14.135783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d6360-71c6-4d34-af8c-f4ff81939d0f None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:14.136355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d6360-71c6-4d34-af8c-f4ff81939d0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:14.138019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d6360-71c6-4d34-af8c-f4ff81939d0f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:14.156242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 88/164] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:14.157506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d6360-71c6-4d34-af8c-f4ff81939d0f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:14.208409 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 77/165] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:14.209352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032de61d-a731-454d-9705-fc73278008da None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:14.385683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:14.408132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d6360-71c6-4d34-af8c-f4ff81939d0f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:14.421956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032de61d-a731-454d-9705-fc73278008da None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:14.488239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ff4e6-16b1-4cc5-affc-a20f4a0aa93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:14.488930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ff4e6-16b1-4cc5-affc-a20f4a0aa93e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:14.489262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ff4e6-16b1-4cc5-affc-a20f4a0aa93e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:14.489537 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ff4e6-16b1-4cc5-affc-a20f4a0aa93e None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:14.490755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ff4e6-16b1-4cc5-affc-a20f4a0aa93e None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:14.493062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ff4e6-16b1-4cc5-affc-a20f4a0aa93e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:14.494631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbce2f-071c-4705-85d2-2f7a9379520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:14.495584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbce2f-071c-4705-85d2-2f7a9379520c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:14.496315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 89/166] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:14 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:14.496530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbce2f-071c-4705-85d2-2f7a9379520c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:14.497003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbce2f-071c-4705-85d2-2f7a9379520c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:14.497594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ff4e6-16b1-4cc5-affc-a20f4a0aa93e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:14.563467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:14.653534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ff4e6-16b1-4cc5-affc-a20f4a0aa93e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:14.693093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbce2f-071c-4705-85d2-2f7a9379520c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:14.693285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a9ea3-9706-4e8a-b6bd-fc9bb2d7bf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:14.693905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a9ea3-9706-4e8a-b6bd-fc9bb2d7bf7b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:14.694178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a9ea3-9706-4e8a-b6bd-fc9bb2d7bf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:14.694447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a9ea3-9706-4e8a-b6bd-fc9bb2d7bf7b None admin] PATH_INFO: `/v3/users/e22fbb4f1849455e917f7324ffd888a3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:14.695714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a9ea3-9706-4e8a-b6bd-fc9bb2d7bf7b None admin] RBAC: Authorizing `identity:get_user(user_id=e22fbb4f1849455e917f7324ffd888a3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:14.695920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbce2f-071c-4705-85d2-2f7a9379520c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:14.697169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a9ea3-9706-4e8a-b6bd-fc9bb2d7bf7b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:14.699195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 90/167] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:14 2022] GET /identity/v3/users/e22fbb4f1849455e917f7324ffd888a3 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:14.700107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a9ea3-9706-4e8a-b6bd-fc9bb2d7bf7b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:14.902249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:14.933620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:14.950246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 78/168] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 741 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:14.951157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbce2f-071c-4705-85d2-2f7a9379520c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:14.956280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a9ea3-9706-4e8a-b6bd-fc9bb2d7bf7b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:15.025755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae866e-d6b0-4478-ae38-f7a1762911fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:15.026655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae866e-d6b0-4478-ae38-f7a1762911fe None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.027273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae866e-d6b0-4478-ae38-f7a1762911fe None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.027273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae866e-d6b0-4478-ae38-f7a1762911fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.027932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae866e-d6b0-4478-ae38-f7a1762911fe None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea, user.id=5d32a6d718b64e35aca00f733d966fd9) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:15.028245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae866e-d6b0-4478-ae38-f7a1762911fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:15.029789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae866e-d6b0-4478-ae38-f7a1762911fe None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:15.048351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 91/169] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:14 2022] GET /identity/v3/role_assignments?user.id=5d32a6d718b64e35aca00f733d966fd9&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 608 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:15.049311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae866e-d6b0-4478-ae38-f7a1762911fe None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:15.144063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbce2f-071c-4705-85d2-2f7a9379520c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:15.214851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5345b1-955b-42a4-adf0-282cc20447c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:15.215404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5345b1-955b-42a4-adf0-282cc20447c9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.215683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5345b1-955b-42a4-adf0-282cc20447c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.215957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5345b1-955b-42a4-adf0-282cc20447c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.268795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:15.271066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae866e-d6b0-4478-ae38-f7a1762911fe None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:15.311632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5345b1-955b-42a4-adf0-282cc20447c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:15.313242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5345b1-955b-42a4-adf0-282cc20447c9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:15.398541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d819f-bf1b-4325-87e1-54caf00d5ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:15.399212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d819f-bf1b-4325-87e1-54caf00d5ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.399503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d819f-bf1b-4325-87e1-54caf00d5ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.399847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d819f-bf1b-4325-87e1-54caf00d5ea6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.427349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d819f-bf1b-4325-87e1-54caf00d5ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-1f7d819f-bf1b-4325-87e1-54caf00d5ea6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:15.433486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:15.435442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 92/170] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:15.436805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d819f-bf1b-4325-87e1-54caf00d5ea6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:15.470748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:15.510202 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 79/171] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:15.512595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1852c-c6d5-453f-8c30-961c6bb54aa8 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.512880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1852c-c6d5-453f-8c30-961c6bb54aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.513162 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1852c-c6d5-453f-8c30-961c6bb54aa8 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.514532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 80/172] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:15.519791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1852c-c6d5-453f-8c30-961c6bb54aa8 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:15.558629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d819f-bf1b-4325-87e1-54caf00d5ea6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:15.598964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acd059-62ef-41f5-95f0-5a1cb79c01de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:15.599661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acd059-62ef-41f5-95f0-5a1cb79c01de None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.599938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acd059-62ef-41f5-95f0-5a1cb79c01de None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.600283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acd059-62ef-41f5-95f0-5a1cb79c01de None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.601586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acd059-62ef-41f5-95f0-5a1cb79c01de None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:15.603358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acd059-62ef-41f5-95f0-5a1cb79c01de None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:15.605367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 93/173] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:15 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:15.606800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc4e60-710d-41e8-b505-fc7c03c83ee1 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.607131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc4e60-710d-41e8-b505-fc7c03c83ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.607400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc4e60-710d-41e8-b505-fc7c03c83ee1 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.608789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 94/174] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:15.610093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec440c-05aa-479b-bfe1-9789760129dc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.610496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec440c-05aa-479b-bfe1-9789760129dc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.610771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec440c-05aa-479b-bfe1-9789760129dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.636555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-51ec440c-05aa-479b-bfe1-9789760129dc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:15.637771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-51ec440c-05aa-479b-bfe1-9789760129dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:15.645561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:15.689249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1852c-c6d5-453f-8c30-961c6bb54aa8 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:15.727221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 95/175] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:15.728018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec440c-05aa-479b-bfe1-9789760129dc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:15.744805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86778812-36bd-4aa8-b146-6be5cf4020a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:15.745562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86778812-36bd-4aa8-b146-6be5cf4020a6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.745852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86778812-36bd-4aa8-b146-6be5cf4020a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.747430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86778812-36bd-4aa8-b146-6be5cf4020a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.749108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86778812-36bd-4aa8-b146-6be5cf4020a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:15.749423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86778812-36bd-4aa8-b146-6be5cf4020a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:15.754832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86778812-36bd-4aa8-b146-6be5cf4020a6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:15.760214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:15.778381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 81/176] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:15.779838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db4db-3695-4631-9f1b-157164ea7ec1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.780118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db4db-3695-4631-9f1b-157164ea7ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.780386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db4db-3695-4631-9f1b-157164ea7ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.818722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-194db4db-3695-4631-9f1b-157164ea7ec1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:15.820749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-194db4db-3695-4631-9f1b-157164ea7ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:15.925691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec440c-05aa-479b-bfe1-9789760129dc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:15.951995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 82/177] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:15.952960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db4db-3695-4631-9f1b-157164ea7ec1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:15.995622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8bd90-d969-408b-8f5d-29f1277e0658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:15.996234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8bd90-d969-408b-8f5d-29f1277e0658 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:15.996512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8bd90-d969-408b-8f5d-29f1277e0658 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:15.996999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8bd90-d969-408b-8f5d-29f1277e0658 None admin] PATH_INFO: `/v3/projects/bd29757566ad433d88cb11652ca31b8a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:15.998416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8bd90-d969-408b-8f5d-29f1277e0658 None admin] RBAC: Authorizing `identity:get_project(project_id=bd29757566ad433d88cb11652ca31b8a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:16.001156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8bd90-d969-408b-8f5d-29f1277e0658 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:16.007041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 96/178] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:15 2022] GET /identity/v3/projects/bd29757566ad433d88cb11652ca31b8a => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:16.008184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8bd90-d969-408b-8f5d-29f1277e0658 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:16.086660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:16.117979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db4db-3695-4631-9f1b-157164ea7ec1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:16.151036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:16.168115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f00ae-2f2e-4976-8788-21d337b4b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:16.169269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8bd90-d969-408b-8f5d-29f1277e0658 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:16.169427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f00ae-2f2e-4976-8788-21d337b4b206 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.169945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f00ae-2f2e-4976-8788-21d337b4b206 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.170395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f00ae-2f2e-4976-8788-21d337b4b206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.171551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f00ae-2f2e-4976-8788-21d337b4b206 None admin] RBAC: Adding query filter params (role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea, user.id=e22fbb4f1849455e917f7324ffd888a3) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:16.172057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f00ae-2f2e-4976-8788-21d337b4b206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:16.175338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f00ae-2f2e-4976-8788-21d337b4b206 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:16.183500 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 83/179] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:15 2022] GET /identity/v3/role_assignments?user.id=e22fbb4f1849455e917f7324ffd888a3&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=799b2f3554434281bf10b2602fbbc09a => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:16.185725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0f576-0483-4307-9d20-7ce029d71359 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.186425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0f576-0483-4307-9d20-7ce029d71359 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.186885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0f576-0483-4307-9d20-7ce029d71359 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.189317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 84/180] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:16.195290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0f576-0483-4307-9d20-7ce029d71359 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:16.229242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14301d1f-52b2-4cdd-b2e0-89a525268da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:16.231485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14301d1f-52b2-4cdd-b2e0-89a525268da0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.231869 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14301d1f-52b2-4cdd-b2e0-89a525268da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.232167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14301d1f-52b2-4cdd-b2e0-89a525268da0 None admin] PATH_INFO: `/v3/users/e22fbb4f1849455e917f7324ffd888a3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.233939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14301d1f-52b2-4cdd-b2e0-89a525268da0 None admin] RBAC: Authorizing `identity:get_user(user_id=e22fbb4f1849455e917f7324ffd888a3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:16.235893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14301d1f-52b2-4cdd-b2e0-89a525268da0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:16.238644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 97/181] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:16 2022] GET /identity/v3/users/e22fbb4f1849455e917f7324ffd888a3 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:16.239891 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14301d1f-52b2-4cdd-b2e0-89a525268da0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:16.341068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0f576-0483-4307-9d20-7ce029d71359 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:16.387796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764da6-aaef-4fa1-bcb2-90f8206dca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:16.388410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764da6-aaef-4fa1-bcb2-90f8206dca15 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.388903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764da6-aaef-4fa1-bcb2-90f8206dca15 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.389144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764da6-aaef-4fa1-bcb2-90f8206dca15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.389912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764da6-aaef-4fa1-bcb2-90f8206dca15 None admin] RBAC: Adding query filter params (role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=bd29757566ad433d88cb11652ca31b8a, user.id=e22fbb4f1849455e917f7324ffd888a3) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:16.390251 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764da6-aaef-4fa1-bcb2-90f8206dca15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:16.391710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764da6-aaef-4fa1-bcb2-90f8206dca15 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:16.395326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:16.395460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14301d1f-52b2-4cdd-b2e0-89a525268da0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:16.400081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 85/182] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:16 2022] GET /identity/v3/role_assignments?user.id=e22fbb4f1849455e917f7324ffd888a3&scope.project.id=bd29757566ad433d88cb11652ca31b8a&role.id=799b2f3554434281bf10b2602fbbc09a => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:16.401790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cdd69-13c0-4de2-ab55-5f6caef6ccf5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.401790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cdd69-13c0-4de2-ab55-5f6caef6ccf5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.402101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cdd69-13c0-4de2-ab55-5f6caef6ccf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.427468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0a0cdd69-13c0-4de2-ab55-5f6caef6ccf5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:16.428769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0a0cdd69-13c0-4de2-ab55-5f6caef6ccf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:16.446431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384b6bc-4da0-4958-b956-94104ec86c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:16.447144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384b6bc-4da0-4958-b956-94104ec86c22 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.447404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384b6bc-4da0-4958-b956-94104ec86c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.447647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384b6bc-4da0-4958-b956-94104ec86c22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.463031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1384b6bc-4da0-4958-b956-94104ec86c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-1384b6bc-4da0-4958-b956-94104ec86c22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:16.469095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:16.470082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:16.470878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 98/183] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:16.472389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7f4f5-348f-4dfd-b009-e22275cb6b55 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.472674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7f4f5-348f-4dfd-b009-e22275cb6b55 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.472956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7f4f5-348f-4dfd-b009-e22275cb6b55 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.475161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 99/184] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:16.475975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7f4f5-348f-4dfd-b009-e22275cb6b55 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:16.499379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:16.538245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 86/185] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:16.539135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cdd69-13c0-4de2-ab55-5f6caef6ccf5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:16.622071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7f4f5-348f-4dfd-b009-e22275cb6b55 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:16.685129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01be9b-1cf3-4978-bd22-cc2dff9adb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:16.686400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01be9b-1cf3-4978-bd22-cc2dff9adb4a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.686722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01be9b-1cf3-4978-bd22-cc2dff9adb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.687039 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01be9b-1cf3-4978-bd22-cc2dff9adb4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.706560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cdd69-13c0-4de2-ab55-5f6caef6ccf5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:16.743290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:16.753852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc87b27-8bfb-4b1d-9112-7d23d707290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:16.754637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc87b27-8bfb-4b1d-9112-7d23d707290b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.768025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc87b27-8bfb-4b1d-9112-7d23d707290b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.768025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc87b27-8bfb-4b1d-9112-7d23d707290b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.772184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc87b27-8bfb-4b1d-9112-7d23d707290b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-5bc87b27-8bfb-4b1d-9112-7d23d707290b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:16.777227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:16.778451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:16.779261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 87/186] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:04:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:16.780012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc87b27-8bfb-4b1d-9112-7d23d707290b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:16.787900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01be9b-1cf3-4978-bd22-cc2dff9adb4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:16.790837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01be9b-1cf3-4978-bd22-cc2dff9adb4a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:16.817757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:16.890081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc87b27-8bfb-4b1d-9112-7d23d707290b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:16.910437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 100/187] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:16.911692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655b7e7-44fa-41e0-8d2a-2bd3721637a9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.911976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655b7e7-44fa-41e0-8d2a-2bd3721637a9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.912249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655b7e7-44fa-41e0-8d2a-2bd3721637a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.931501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a655b7e7-44fa-41e0-8d2a-2bd3721637a9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:16.931843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce586cd-16cb-4d2f-b3e8-d9e0ccad354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:16.932458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce586cd-16cb-4d2f-b3e8-d9e0ccad354c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:16.932738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce586cd-16cb-4d2f-b3e8-d9e0ccad354c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:16.932974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a655b7e7-44fa-41e0-8d2a-2bd3721637a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:16.933114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce586cd-16cb-4d2f-b3e8-d9e0ccad354c None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:16.933898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce586cd-16cb-4d2f-b3e8-d9e0ccad354c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:16.934476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce586cd-16cb-4d2f-b3e8-d9e0ccad354c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:16.935881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce586cd-16cb-4d2f-b3e8-d9e0ccad354c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:16.946552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 88/188] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:16.947675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce586cd-16cb-4d2f-b3e8-d9e0ccad354c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:17.017718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:17.024200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 101/189] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:17.024901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655b7e7-44fa-41e0-8d2a-2bd3721637a9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:17.081073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce586cd-16cb-4d2f-b3e8-d9e0ccad354c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:17.093645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:17.124433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524b5b2-2330-4111-8d8b-512ee31c06c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:17.125166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524b5b2-2330-4111-8d8b-512ee31c06c1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:17.125454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524b5b2-2330-4111-8d8b-512ee31c06c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:17.125975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524b5b2-2330-4111-8d8b-512ee31c06c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:17.126446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524b5b2-2330-4111-8d8b-512ee31c06c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:17.126759 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524b5b2-2330-4111-8d8b-512ee31c06c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:17.128827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524b5b2-2330-4111-8d8b-512ee31c06c1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:17.138200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 89/190] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:17.143974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524b5b2-2330-4111-8d8b-512ee31c06c1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:17.166684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655b7e7-44fa-41e0-8d2a-2bd3721637a9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:17.217832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae489c-5329-402d-a136-32eac239b43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:17.218961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae489c-5329-402d-a136-32eac239b43e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:17.219437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae489c-5329-402d-a136-32eac239b43e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:17.219915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae489c-5329-402d-a136-32eac239b43e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:17.222042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae489c-5329-402d-a136-32eac239b43e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:17.224783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae489c-5329-402d-a136-32eac239b43e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:17.232526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 102/191] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:17.232526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae489c-5329-402d-a136-32eac239b43e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:17.272649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524b5b2-2330-4111-8d8b-512ee31c06c1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:17.292347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:17.318951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce761cf1-d642-47ed-90a2-bb4c7e57750f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:17.319641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce761cf1-d642-47ed-90a2-bb4c7e57750f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:17.319929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce761cf1-d642-47ed-90a2-bb4c7e57750f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:17.320210 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce761cf1-d642-47ed-90a2-bb4c7e57750f None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:17.321693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce761cf1-d642-47ed-90a2-bb4c7e57750f None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:17.323278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce761cf1-d642-47ed-90a2-bb4c7e57750f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:17.325368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 90/192] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:17 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:17.326087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce761cf1-d642-47ed-90a2-bb4c7e57750f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:17.374407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae489c-5329-402d-a136-32eac239b43e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:17.389933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:17.415139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c366ed-d4d1-496b-8620-6f814660f9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:17.415772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c366ed-d4d1-496b-8620-6f814660f9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:17.415976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c366ed-d4d1-496b-8620-6f814660f9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:17.416371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c366ed-d4d1-496b-8620-6f814660f9c2 None admin] PATH_INFO: `/v3/users/725fd21c3867436692a2e10ed4770378` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:17.417445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c366ed-d4d1-496b-8620-6f814660f9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=725fd21c3867436692a2e10ed4770378)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:17.418971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c366ed-d4d1-496b-8620-6f814660f9c2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:17.420643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 103/193] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:17 2022] GET /identity/v3/users/725fd21c3867436692a2e10ed4770378 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:17.421425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c366ed-d4d1-496b-8620-6f814660f9c2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:17.448616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce761cf1-d642-47ed-90a2-bb4c7e57750f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:17.490403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcd10e-898d-46d7-9a9e-557592e84a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:17.491081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcd10e-898d-46d7-9a9e-557592e84a55 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:17.491387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcd10e-898d-46d7-9a9e-557592e84a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:17.491660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcd10e-898d-46d7-9a9e-557592e84a55 None admin] PATH_INFO: `/v3/users/ad316a97ff1a405281301cf0f697665f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:17.507479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcd10e-898d-46d7-9a9e-557592e84a55 None admin] RBAC: Authorizing `identity:get_user(user_id=ad316a97ff1a405281301cf0f697665f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:17.508986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcd10e-898d-46d7-9a9e-557592e84a55 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:17.510757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 91/194] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:17 2022] GET /identity/v3/users/ad316a97ff1a405281301cf0f697665f => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:17.511567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcd10e-898d-46d7-9a9e-557592e84a55 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:17.543315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c366ed-d4d1-496b-8620-6f814660f9c2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:17.556396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:17.584291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b3442-1550-4225-aaae-b9409588d391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:17.585207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b3442-1550-4225-aaae-b9409588d391 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:17.585476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b3442-1550-4225-aaae-b9409588d391 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:17.585755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b3442-1550-4225-aaae-b9409588d391 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:17.586596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b3442-1550-4225-aaae-b9409588d391 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:17.588622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b3442-1550-4225-aaae-b9409588d391 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:17.628073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcd10e-898d-46d7-9a9e-557592e84a55 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:17.665573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21664a5d-2eb6-4494-8f7b-214d890cf16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:17.666166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21664a5d-2eb6-4494-8f7b-214d890cf16e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:17.666442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21664a5d-2eb6-4494-8f7b-214d890cf16e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:17.666716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21664a5d-2eb6-4494-8f7b-214d890cf16e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:17.679772 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21664a5d-2eb6-4494-8f7b-214d890cf16e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-21664a5d-2eb6-4494-8f7b-214d890cf16e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:17.685079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:17.687611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 92/195] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:17.688928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21664a5d-2eb6-4494-8f7b-214d890cf16e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:17.700228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 104/196] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:17 2022] POST /identity/v3/users => generated 297 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:17.704731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b3442-1550-4225-aaae-b9409588d391 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:17.752849 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:17.835288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b3442-1550-4225-aaae-b9409588d391 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:17.846909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:17.852114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21664a5d-2eb6-4494-8f7b-214d890cf16e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:17.877071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5de30b-2ae6-4960-9da3-8df22429bd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:17.878308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5de30b-2ae6-4960-9da3-8df22429bd18 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:17.878594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5de30b-2ae6-4960-9da3-8df22429bd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:17.878872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5de30b-2ae6-4960-9da3-8df22429bd18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:17.897773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d9e4c-43b1-4472-8dbf-32194b8c46ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:17.898445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d9e4c-43b1-4472-8dbf-32194b8c46ef None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:17.898733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d9e4c-43b1-4472-8dbf-32194b8c46ef None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:17.899009 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d9e4c-43b1-4472-8dbf-32194b8c46ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:17.899776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d9e4c-43b1-4472-8dbf-32194b8c46ef None admin] RBAC: Adding query filter params (role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea, user.id=e22fbb4f1849455e917f7324ffd888a3) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:17.900099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d9e4c-43b1-4472-8dbf-32194b8c46ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:17.902063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d9e4c-43b1-4472-8dbf-32194b8c46ef None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:17.909692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 93/197] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:17 2022] GET /identity/v3/role_assignments?user.id=e22fbb4f1849455e917f7324ffd888a3&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:17.914929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d9e4c-43b1-4472-8dbf-32194b8c46ef None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:18.007892 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5de30b-2ae6-4960-9da3-8df22429bd18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:18.010444 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5de30b-2ae6-4960-9da3-8df22429bd18 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:18.075712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:18.125003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d9e4c-43b1-4472-8dbf-32194b8c46ef None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:18.184256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aea5a2-f761-4168-b5f9-187c521c523b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:18.185468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aea5a2-f761-4168-b5f9-187c521c523b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:18.186868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aea5a2-f761-4168-b5f9-187c521c523b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:18.186868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aea5a2-f761-4168-b5f9-187c521c523b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:18.204679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 105/198] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:18.206730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5de30b-2ae6-4960-9da3-8df22429bd18 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:18.261006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:18.336115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aea5a2-f761-4168-b5f9-187c521c523b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:18.336115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aea5a2-f761-4168-b5f9-187c521c523b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:18.377783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5de30b-2ae6-4960-9da3-8df22429bd18 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:18.428389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee050ba-65de-4643-9dfd-483881e5d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:18.429036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee050ba-65de-4643-9dfd-483881e5d239 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:18.429314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee050ba-65de-4643-9dfd-483881e5d239 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:18.430336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee050ba-65de-4643-9dfd-483881e5d239 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:18.430564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee050ba-65de-4643-9dfd-483881e5d239 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:18.430815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee050ba-65de-4643-9dfd-483881e5d239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:18.432465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee050ba-65de-4643-9dfd-483881e5d239 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:18.435476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:18.443157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 106/199] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:18.444400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee050ba-65de-4643-9dfd-483881e5d239 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:18.478222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 94/200] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:18.479567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aea5a2-f761-4168-b5f9-187c521c523b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:18.554557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:18.601352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aea5a2-f761-4168-b5f9-187c521c523b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:18.610220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee050ba-65de-4643-9dfd-483881e5d239 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:18.657047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181305e-040a-4b88-8787-7a964eae0714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:18.657689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181305e-040a-4b88-8787-7a964eae0714 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:18.658034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181305e-040a-4b88-8787-7a964eae0714 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:18.658271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181305e-040a-4b88-8787-7a964eae0714 None admin] PATH_INFO: `/v3/users/ad316a97ff1a405281301cf0f697665f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:18.660036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181305e-040a-4b88-8787-7a964eae0714 None admin] RBAC: Authorizing `identity:get_user(user_id=ad316a97ff1a405281301cf0f697665f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:18.660795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645042-fc11-4d76-a47f-ac87c9c534f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:18.661511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181305e-040a-4b88-8787-7a964eae0714 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:18.661779 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645042-fc11-4d76-a47f-ac87c9c534f6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:18.662249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645042-fc11-4d76-a47f-ac87c9c534f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:18.662769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645042-fc11-4d76-a47f-ac87c9c534f6 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:18.663214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 95/201] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:18 2022] GET /identity/v3/users/ad316a97ff1a405281301cf0f697665f => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:18.663999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181305e-040a-4b88-8787-7a964eae0714 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:18.664767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69645042-fc11-4d76-a47f-ac87c9c534f6 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:18.666316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69645042-fc11-4d76-a47f-ac87c9c534f6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:18.669289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 107/202] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:18 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:18.675135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645042-fc11-4d76-a47f-ac87c9c534f6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:18.796017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:18.850000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181305e-040a-4b88-8787-7a964eae0714 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:18.850333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645042-fc11-4d76-a47f-ac87c9c534f6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:18.893697 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:18.898006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c1aaf-b7da-42e5-abb4-0a82a39a3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:18.898870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c1aaf-b7da-42e5-abb4-0a82a39a3b8b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:18.899309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c1aaf-b7da-42e5-abb4-0a82a39a3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:18.899615 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c1aaf-b7da-42e5-abb4-0a82a39a3b8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:18.902417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230b7b8-3e2e-4eea-9ca1-6e3a7d706a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:18.903575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230b7b8-3e2e-4eea-9ca1-6e3a7d706a27 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:18.904162 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230b7b8-3e2e-4eea-9ca1-6e3a7d706a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:18.907216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230b7b8-3e2e-4eea-9ca1-6e3a7d706a27 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:18.907216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230b7b8-3e2e-4eea-9ca1-6e3a7d706a27 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:18.909223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230b7b8-3e2e-4eea-9ca1-6e3a7d706a27 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:18.911461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 96/203] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:18 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:18.912660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230b7b8-3e2e-4eea-9ca1-6e3a7d706a27 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:19.002314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c1aaf-b7da-42e5-abb4-0a82a39a3b8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:19.004532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c1aaf-b7da-42e5-abb4-0a82a39a3b8b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:19.080748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230b7b8-3e2e-4eea-9ca1-6e3a7d706a27 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:19.122689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:19.125901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9d226-acce-4761-816a-210c62cdb259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:19.127233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9d226-acce-4761-816a-210c62cdb259 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:19.127538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9d226-acce-4761-816a-210c62cdb259 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:19.127824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9d226-acce-4761-816a-210c62cdb259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:19.128620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9d226-acce-4761-816a-210c62cdb259 None admin] RBAC: Adding query filter params (role.id=0f2f4356fa294a7db98c5636fe2d390d, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea, user.id=725fd21c3867436692a2e10ed4770378) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:19.129066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9d226-acce-4761-816a-210c62cdb259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:19.131061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9d226-acce-4761-816a-210c62cdb259 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:19.139184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 97/204] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:18 2022] GET /identity/v3/role_assignments?user.id=725fd21c3867436692a2e10ed4770378&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=0f2f4356fa294a7db98c5636fe2d390d => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:19.139903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9d226-acce-4761-816a-210c62cdb259 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:19.269729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 108/205] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:19.276353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c1aaf-b7da-42e5-abb4-0a82a39a3b8b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:19.326934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:19.404600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9d226-acce-4761-816a-210c62cdb259 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:19.453444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076145f-6aaa-456e-a8f7-c6fe3cade5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:19.454608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076145f-6aaa-456e-a8f7-c6fe3cade5bc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:19.455072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076145f-6aaa-456e-a8f7-c6fe3cade5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:19.455571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076145f-6aaa-456e-a8f7-c6fe3cade5bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:19.456516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076145f-6aaa-456e-a8f7-c6fe3cade5bc None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=858cb936469e4382b567530855fa7507, user.id=ad316a97ff1a405281301cf0f697665f) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:19.457007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076145f-6aaa-456e-a8f7-c6fe3cade5bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:19.458805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076145f-6aaa-456e-a8f7-c6fe3cade5bc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:19.466768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 98/206] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:19 2022] GET /identity/v3/role_assignments?user.id=ad316a97ff1a405281301cf0f697665f&scope.project.id=858cb936469e4382b567530855fa7507&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:19.495005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c1aaf-b7da-42e5-abb4-0a82a39a3b8b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:19.538153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df931cd-b65b-48b6-8ddd-0082fe7b5b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:19.538831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df931cd-b65b-48b6-8ddd-0082fe7b5b9a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:19.539116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df931cd-b65b-48b6-8ddd-0082fe7b5b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:19.539504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df931cd-b65b-48b6-8ddd-0082fe7b5b9a None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:19.540963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df931cd-b65b-48b6-8ddd-0082fe7b5b9a None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:19.542675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df931cd-b65b-48b6-8ddd-0082fe7b5b9a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:19.544717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 109/207] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:19 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:19.554376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076145f-6aaa-456e-a8f7-c6fe3cade5bc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:19.695989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076145f-6aaa-456e-a8f7-c6fe3cade5bc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:19.726410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:19.734026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7a9e0-2797-4388-a12b-4922bd1630e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:19.734732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7a9e0-2797-4388-a12b-4922bd1630e7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:19.735009 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7a9e0-2797-4388-a12b-4922bd1630e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:19.735281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7a9e0-2797-4388-a12b-4922bd1630e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:19.736024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7a9e0-2797-4388-a12b-4922bd1630e7 None admin] RBAC: Adding query filter params (role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=858cb936469e4382b567530855fa7507, user.id=ad316a97ff1a405281301cf0f697665f) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:19.736402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7a9e0-2797-4388-a12b-4922bd1630e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:19.737973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7a9e0-2797-4388-a12b-4922bd1630e7 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:19.744178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 99/208] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:19 2022] GET /identity/v3/role_assignments?user.id=ad316a97ff1a405281301cf0f697665f&scope.project.id=858cb936469e4382b567530855fa7507&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:20.505045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adab6fa-f6d0-4281-9d0d-866f1b07c3fd None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:20.505045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adab6fa-f6d0-4281-9d0d-866f1b07c3fd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:20.505045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adab6fa-f6d0-4281-9d0d-866f1b07c3fd None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:20.505045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 110/209] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:20.513845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e97b9-2e01-471f-b8dd-fd8f67701d1c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:20.514793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e97b9-2e01-471f-b8dd-fd8f67701d1c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:20.515307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e97b9-2e01-471f-b8dd-fd8f67701d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:20.543038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-676e97b9-2e01-471f-b8dd-fd8f67701d1c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:20.544586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-676e97b9-2e01-471f-b8dd-fd8f67701d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:20.623098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587dfd8-ff01-48ba-bd82-8d8809e07e38 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:20.623394 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587dfd8-ff01-48ba-bd82-8d8809e07e38 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:20.623608 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587dfd8-ff01-48ba-bd82-8d8809e07e38 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:20.625302 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 111/210] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:04:20.632607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d55c0-f1d6-4167-ba4c-c986c3549d18 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:20.633005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d55c0-f1d6-4167-ba4c-c986c3549d18 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:20.633252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d55c0-f1d6-4167-ba4c-c986c3549d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:20.641097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 100/211] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:20.656607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7e9d55c0-f1d6-4167-ba4c-c986c3549d18 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:20.657997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7e9d55c0-f1d6-4167-ba4c-c986c3549d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:20.694433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:20.743944 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 112/212] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:21.032156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e97b9-2e01-471f-b8dd-fd8f67701d1c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:21.038943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d55c0-f1d6-4167-ba4c-c986c3549d18 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:21.185706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d55c0-f1d6-4167-ba4c-c986c3549d18 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:21.198881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e97b9-2e01-471f-b8dd-fd8f67701d1c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:21.234406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:21.244330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d67b8-650c-4b7b-abb5-542081ac5e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:21.245376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d67b8-650c-4b7b-abb5-542081ac5e83 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:21.245907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d67b8-650c-4b7b-abb5-542081ac5e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:21.246384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d67b8-650c-4b7b-abb5-542081ac5e83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:21.253440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dcfae-b649-418f-b7d6-dfd72934e367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:21.254140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dcfae-b649-418f-b7d6-dfd72934e367 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:21.254421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dcfae-b649-418f-b7d6-dfd72934e367 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:21.254727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dcfae-b649-418f-b7d6-dfd72934e367 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:21.255880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dcfae-b649-418f-b7d6-dfd72934e367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:21.257553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dcfae-b649-418f-b7d6-dfd72934e367 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:21.259819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 101/213] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:21.262909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d67b8-650c-4b7b-abb5-542081ac5e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:21.266840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dcfae-b649-418f-b7d6-dfd72934e367 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-dd2d67b8-650c-4b7b-abb5-542081ac5e83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:21.268666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:21.269544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:21.270412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:21.272994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 113/214] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:04:21.279301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d67b8-650c-4b7b-abb5-542081ac5e83 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:21.315910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:21.412397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dcfae-b649-418f-b7d6-dfd72934e367 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:21.426167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d67b8-650c-4b7b-abb5-542081ac5e83 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:21.453524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5c7ba-0d1a-46f7-8f55-079079a1f274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:21.454278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5c7ba-0d1a-46f7-8f55-079079a1f274 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:21.454562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5c7ba-0d1a-46f7-8f55-079079a1f274 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:21.454992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5c7ba-0d1a-46f7-8f55-079079a1f274 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:21.456292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5c7ba-0d1a-46f7-8f55-079079a1f274 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:21.457864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5c7ba-0d1a-46f7-8f55-079079a1f274 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:21.475181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e7680-6ee5-4ee5-8f1b-d65e1a0ae8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:21.475933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e7680-6ee5-4ee5-8f1b-d65e1a0ae8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:21.476283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e7680-6ee5-4ee5-8f1b-d65e1a0ae8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:21.476614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e7680-6ee5-4ee5-8f1b-d65e1a0ae8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:21.477376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e7680-6ee5-4ee5-8f1b-d65e1a0ae8c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:21.477752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e7680-6ee5-4ee5-8f1b-d65e1a0ae8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:21.479445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e7680-6ee5-4ee5-8f1b-d65e1a0ae8c7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:21.489908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 114/215] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:21.496110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e7680-6ee5-4ee5-8f1b-d65e1a0ae8c7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:21.604407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 102/216] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:21 2022] POST /identity/v3/users => generated 297 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:21.611532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:21.633190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e7680-6ee5-4ee5-8f1b-d65e1a0ae8c7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:21.677774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350392a-1e0a-4381-a79a-7b0caa81ddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:21.678770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350392a-1e0a-4381-a79a-7b0caa81ddfb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:21.679224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350392a-1e0a-4381-a79a-7b0caa81ddfb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:21.679713 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350392a-1e0a-4381-a79a-7b0caa81ddfb None admin] PATH_INFO: `/v3/users/e2f1b018728f4931a9ee0dbdf0917789` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:21.695679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220eb74-e76a-4ee6-841a-ccd8daff3ced None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:21.695972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220eb74-e76a-4ee6-841a-ccd8daff3ced None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:21.696244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220eb74-e76a-4ee6-841a-ccd8daff3ced None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:21.698143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 103/217] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:04:21.699925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350392a-1e0a-4381-a79a-7b0caa81ddfb None admin] RBAC: Authorizing `identity:get_user(user_id=e2f1b018728f4931a9ee0dbdf0917789)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:21.701655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350392a-1e0a-4381-a79a-7b0caa81ddfb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:21.703712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8629a-b764-49f9-bd35-65a12945df7b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:21.704041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8629a-b764-49f9-bd35-65a12945df7b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:21.704276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8629a-b764-49f9-bd35-65a12945df7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:21.704656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 115/218] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:21 2022] GET /identity/v3/users/e2f1b018728f4931a9ee0dbdf0917789 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:21.712079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350392a-1e0a-4381-a79a-7b0caa81ddfb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:21.730545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-42b8629a-b764-49f9-bd35-65a12945df7b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:21.731826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-42b8629a-b764-49f9-bd35-65a12945df7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:21.782315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:21.825949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 104/219] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:21.849342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350392a-1e0a-4381-a79a-7b0caa81ddfb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:21.888382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f528a6-5a71-49cd-9c0b-d4cc24fc86c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:21.888922 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f528a6-5a71-49cd-9c0b-d4cc24fc86c8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:21.889200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f528a6-5a71-49cd-9c0b-d4cc24fc86c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:21.889534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f528a6-5a71-49cd-9c0b-d4cc24fc86c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:21.907143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:21.983152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f528a6-5a71-49cd-9c0b-d4cc24fc86c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:21.985963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f528a6-5a71-49cd-9c0b-d4cc24fc86c8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:22.113795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8629a-b764-49f9-bd35-65a12945df7b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:22.168019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 116/220] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:22.169667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14463209-2e1b-4dc7-b8ca-003d60a9f6c4 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:22.170199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14463209-2e1b-4dc7-b8ca-003d60a9f6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:22.170714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14463209-2e1b-4dc7-b8ca-003d60a9f6c4 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:22.176443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 117/221] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:22.178923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b217c-ca6b-4e2b-bd3b-e87c9709a7de None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:22.179478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b217c-ca6b-4e2b-bd3b-e87c9709a7de None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:22.179955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b217c-ca6b-4e2b-bd3b-e87c9709a7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:22.202120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-718b217c-ca6b-4e2b-bd3b-e87c9709a7de None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:22.204592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-718b217c-ca6b-4e2b-bd3b-e87c9709a7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:22.255189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:22.271278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8629a-b764-49f9-bd35-65a12945df7b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:22.296603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 118/222] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:22.298360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b217c-ca6b-4e2b-bd3b-e87c9709a7de None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:22.320214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5ed8b-764b-4c3e-aa23-eb7c2ded14bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:22.321909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5ed8b-764b-4c3e-aa23-eb7c2ded14bc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:22.321909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5ed8b-764b-4c3e-aa23-eb7c2ded14bc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:22.322080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5ed8b-764b-4c3e-aa23-eb7c2ded14bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:22.343092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5ed8b-764b-4c3e-aa23-eb7c2ded14bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-3ee5ed8b-764b-4c3e-aa23-eb7c2ded14bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:22.349768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:22.350862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:22.351781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 105/223] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:04:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:22.358031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5ed8b-764b-4c3e-aa23-eb7c2ded14bc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:22.402792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:22.452924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b217c-ca6b-4e2b-bd3b-e87c9709a7de None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:22.486910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5ed8b-764b-4c3e-aa23-eb7c2ded14bc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:22.499135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97032ae-f491-472f-ab6b-550b79757fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:22.500134 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97032ae-f491-472f-ab6b-550b79757fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:22.500662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97032ae-f491-472f-ab6b-550b79757fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:22.501138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97032ae-f491-472f-ab6b-550b79757fe5 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:22.502624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97032ae-f491-472f-ab6b-550b79757fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:22.504304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97032ae-f491-472f-ab6b-550b79757fe5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:22.506363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 119/224] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:22 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:22.514330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97032ae-f491-472f-ab6b-550b79757fe5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:22.537563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ebe30-3fc1-4bd5-a5ad-e00b16be17e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:22.539846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ebe30-3fc1-4bd5-a5ad-e00b16be17e9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:22.541160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ebe30-3fc1-4bd5-a5ad-e00b16be17e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:22.541828 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ebe30-3fc1-4bd5-a5ad-e00b16be17e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:22.544155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ebe30-3fc1-4bd5-a5ad-e00b16be17e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:22.545484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ebe30-3fc1-4bd5-a5ad-e00b16be17e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:22.550383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ebe30-3fc1-4bd5-a5ad-e00b16be17e9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:22.571045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 106/225] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:22.578517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ebe30-3fc1-4bd5-a5ad-e00b16be17e9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:22.594283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:22.672585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97032ae-f491-472f-ab6b-550b79757fe5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:22.724794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e0dfb-65ec-4b1c-94e8-a6d2f4848ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:22.725321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e0dfb-65ec-4b1c-94e8-a6d2f4848ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:22.725594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e0dfb-65ec-4b1c-94e8-a6d2f4848ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:22.725883 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e0dfb-65ec-4b1c-94e8-a6d2f4848ef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:22.726637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e0dfb-65ec-4b1c-94e8-a6d2f4848ef6 None admin] RBAC: Adding query filter params (role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=858cb936469e4382b567530855fa7507, user.id=e2f1b018728f4931a9ee0dbdf0917789) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:22.726941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e0dfb-65ec-4b1c-94e8-a6d2f4848ef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:22.728405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e0dfb-65ec-4b1c-94e8-a6d2f4848ef6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:22.735268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 120/226] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:22 2022] GET /identity/v3/role_assignments?user.id=e2f1b018728f4931a9ee0dbdf0917789&scope.project.id=858cb936469e4382b567530855fa7507&role.id=799b2f3554434281bf10b2602fbbc09a => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:22.736142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e0dfb-65ec-4b1c-94e8-a6d2f4848ef6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:22.775493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:22.792872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ebe30-3fc1-4bd5-a5ad-e00b16be17e9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:22.889253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e0dfb-65ec-4b1c-94e8-a6d2f4848ef6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:22.894647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739b127-ec7c-4898-b55e-8d437c709fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:22.895295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739b127-ec7c-4898-b55e-8d437c709fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:22.895571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739b127-ec7c-4898-b55e-8d437c709fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:22.895841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739b127-ec7c-4898-b55e-8d437c709fc2 None admin] PATH_INFO: `/v3/users/ad316a97ff1a405281301cf0f697665f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:22.897127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739b127-ec7c-4898-b55e-8d437c709fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=ad316a97ff1a405281301cf0f697665f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:22.898617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739b127-ec7c-4898-b55e-8d437c709fc2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:22.900451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:22.901323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 107/227] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:22 2022] GET /identity/v3/users/ad316a97ff1a405281301cf0f697665f => generated 283 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:22.907321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739b127-ec7c-4898-b55e-8d437c709fc2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:22.974248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea9998-5c86-487f-9b50-386acf289465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:22.974983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea9998-5c86-487f-9b50-386acf289465 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:22.975182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea9998-5c86-487f-9b50-386acf289465 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:22.975532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea9998-5c86-487f-9b50-386acf289465 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:23.004845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea9998-5c86-487f-9b50-386acf289465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-12ea9998-5c86-487f-9b50-386acf289465 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:23.019361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:23.020336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:23.021138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 121/228] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:23.040463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea9998-5c86-487f-9b50-386acf289465 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:23.052096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739b127-ec7c-4898-b55e-8d437c709fc2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:23.097031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40023082-bcc5-4694-bc6e-a2eab342b209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:23.098225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40023082-bcc5-4694-bc6e-a2eab342b209 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:23.098706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40023082-bcc5-4694-bc6e-a2eab342b209 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:23.099204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40023082-bcc5-4694-bc6e-a2eab342b209 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:23.100741 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40023082-bcc5-4694-bc6e-a2eab342b209 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:23.102443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40023082-bcc5-4694-bc6e-a2eab342b209 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:23.104479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 108/229] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:22 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:23.112449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40023082-bcc5-4694-bc6e-a2eab342b209 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:23.164701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea9998-5c86-487f-9b50-386acf289465 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:23.171045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:23.207350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde853a9-cd4d-4697-bc44-a6f01e1c4444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:23.208387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde853a9-cd4d-4697-bc44-a6f01e1c4444 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:23.208887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde853a9-cd4d-4697-bc44-a6f01e1c4444 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:23.209452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde853a9-cd4d-4697-bc44-a6f01e1c4444 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:23.210358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde853a9-cd4d-4697-bc44-a6f01e1c4444 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:23.211771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde853a9-cd4d-4697-bc44-a6f01e1c4444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:23.213504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde853a9-cd4d-4697-bc44-a6f01e1c4444 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:23.224706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 122/230] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:23.235044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde853a9-cd4d-4697-bc44-a6f01e1c4444 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:23.294700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:23.299299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40023082-bcc5-4694-bc6e-a2eab342b209 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:23.370925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635dd99-3a05-483e-8aad-72e333cc88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:23.371924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635dd99-3a05-483e-8aad-72e333cc88ad None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:23.372273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635dd99-3a05-483e-8aad-72e333cc88ad None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:23.372553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635dd99-3a05-483e-8aad-72e333cc88ad None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507/users/ad316a97ff1a405281301cf0f697665f/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:23.378026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde853a9-cd4d-4697-bc44-a6f01e1c4444 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:23.379407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635dd99-3a05-483e-8aad-72e333cc88ad None admin] RBAC: Authorizing `identity:create_grant(project_id=858cb936469e4382b567530855fa7507, user_id=ad316a97ff1a405281301cf0f697665f, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:23.381618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635dd99-3a05-483e-8aad-72e333cc88ad None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:23.420035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cff3c-8dc4-4104-9d2a-7c6f8b0d55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:23.421123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cff3c-8dc4-4104-9d2a-7c6f8b0d55d8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:23.421601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cff3c-8dc4-4104-9d2a-7c6f8b0d55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:23.422111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cff3c-8dc4-4104-9d2a-7c6f8b0d55d8 None admin] PATH_INFO: `/v3/users/ad316a97ff1a405281301cf0f697665f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:23.423935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cff3c-8dc4-4104-9d2a-7c6f8b0d55d8 None admin] RBAC: Authorizing `identity:get_user(user_id=ad316a97ff1a405281301cf0f697665f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:23.426478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cff3c-8dc4-4104-9d2a-7c6f8b0d55d8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:23.429091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 123/231] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:23 2022] GET /identity/v3/users/ad316a97ff1a405281301cf0f697665f => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:23.435148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cff3c-8dc4-4104-9d2a-7c6f8b0d55d8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:23.527364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 109/232] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:04:23 2022] PUT /identity/v3/projects/858cb936469e4382b567530855fa7507/users/ad316a97ff1a405281301cf0f697665f/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:23.556446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:23.570709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cff3c-8dc4-4104-9d2a-7c6f8b0d55d8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:23.620929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf099ec-73a3-442f-a802-9891b0852792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:23.621597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf099ec-73a3-442f-a802-9891b0852792 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:23.621910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf099ec-73a3-442f-a802-9891b0852792 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:23.622175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf099ec-73a3-442f-a802-9891b0852792 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:23.623378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf099ec-73a3-442f-a802-9891b0852792 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:23.624935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf099ec-73a3-442f-a802-9891b0852792 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:23.626655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 124/233] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:23 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:23.633597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635dd99-3a05-483e-8aad-72e333cc88ad None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:23.777337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:23.788890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635dd99-3a05-483e-8aad-72e333cc88ad None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:23.826633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0563c-db50-48f1-ba84-55a6b47e96ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:23.827694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0563c-db50-48f1-ba84-55a6b47e96ca None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:23.828146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0563c-db50-48f1-ba84-55a6b47e96ca None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:23.828585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0563c-db50-48f1-ba84-55a6b47e96ca None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507/users/ad316a97ff1a405281301cf0f697665f/roles/e08fcbbf38b344f48f71ed9ceb314293` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:23.831149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0563c-db50-48f1-ba84-55a6b47e96ca None admin] RBAC: Authorizing `identity:create_grant(project_id=858cb936469e4382b567530855fa7507, user_id=ad316a97ff1a405281301cf0f697665f, role_id=e08fcbbf38b344f48f71ed9ceb314293)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:23.832839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0563c-db50-48f1-ba84-55a6b47e96ca None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:23.987595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 110/234] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:04:23 2022] PUT /identity/v3/projects/858cb936469e4382b567530855fa7507/users/ad316a97ff1a405281301cf0f697665f/roles/e08fcbbf38b344f48f71ed9ceb314293 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:24.296628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92863b35-8101-41d6-af3a-9daf35cda7e8 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:24.297175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92863b35-8101-41d6-af3a-9daf35cda7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:24.297175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92863b35-8101-41d6-af3a-9daf35cda7e8 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:24.298921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 125/235] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:24.304653 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c21e4-72da-47f4-8102-a8b2862d655a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:24.304940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c21e4-72da-47f4-8102-a8b2862d655a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:24.305217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c21e4-72da-47f4-8102-a8b2862d655a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:24.328382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-327c21e4-72da-47f4-8102-a8b2862d655a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:24.329601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-327c21e4-72da-47f4-8102-a8b2862d655a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:24.428241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 111/236] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:24.471622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea148564-a44d-4564-9ece-25228c3ddd51 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:24.471954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea148564-a44d-4564-9ece-25228c3ddd51 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:24.472178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea148564-a44d-4564-9ece-25228c3ddd51 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:24.474058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 126/237] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:24.492132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad477a86-8ec3-4068-9e36-51e4fb911081 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:24.492436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad477a86-8ec3-4068-9e36-51e4fb911081 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:24.492698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad477a86-8ec3-4068-9e36-51e4fb911081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:24.519231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ad477a86-8ec3-4068-9e36-51e4fb911081 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:24.520463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ad477a86-8ec3-4068-9e36-51e4fb911081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:24.619375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 112/238] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:24.866298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea148564-a44d-4564-9ece-25228c3ddd51 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:24.960428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad477a86-8ec3-4068-9e36-51e4fb911081 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:24.987798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:25.038329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea148564-a44d-4564-9ece-25228c3ddd51 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:25.089969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08670c-0823-481c-a0e5-8d48e3d52341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:25.090626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08670c-0823-481c-a0e5-8d48e3d52341 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.090907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08670c-0823-481c-a0e5-8d48e3d52341 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.091181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08670c-0823-481c-a0e5-8d48e3d52341 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:25.093083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08670c-0823-481c-a0e5-8d48e3d52341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:25.095341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08670c-0823-481c-a0e5-8d48e3d52341 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:25.099320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 127/239] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:25.100488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b130828-9d4d-4193-80a8-b6ea0b4202fa None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.100769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b130828-9d4d-4193-80a8-b6ea0b4202fa None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.101042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b130828-9d4d-4193-80a8-b6ea0b4202fa None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:25.102528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 128/240] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:25.107575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b130828-9d4d-4193-80a8-b6ea0b4202fa None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:25.141124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:25.147497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad477a86-8ec3-4068-9e36-51e4fb911081 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:25.187796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af5e3-e35a-4d69-9e79-a2133a62279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:25.188878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af5e3-e35a-4d69-9e79-a2133a62279f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.189590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af5e3-e35a-4d69-9e79-a2133a62279f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.190590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af5e3-e35a-4d69-9e79-a2133a62279f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:25.206637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22af5e3-e35a-4d69-9e79-a2133a62279f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-f22af5e3-e35a-4d69-9e79-a2133a62279f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:25.214752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:25.215629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:25.216427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:25.220086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 113/241] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:04:25.222341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723725e3-1cb4-4e44-bb0e-96fcb423608d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.222905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723725e3-1cb4-4e44-bb0e-96fcb423608d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.223446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723725e3-1cb4-4e44-bb0e-96fcb423608d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:25.227952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b130828-9d4d-4193-80a8-b6ea0b4202fa None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:25.258803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-723725e3-1cb4-4e44-bb0e-96fcb423608d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:25.260919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-723725e3-1cb4-4e44-bb0e-96fcb423608d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:25.267358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd1227-fa23-436a-b648-9758239c6818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:25.268125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd1227-fa23-436a-b648-9758239c6818 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.268377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd1227-fa23-436a-b648-9758239c6818 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.268637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd1227-fa23-436a-b648-9758239c6818 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:25.269294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd1227-fa23-436a-b648-9758239c6818 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:25.270888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd1227-fa23-436a-b648-9758239c6818 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:25.350161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 114/242] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:25.350161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723725e3-1cb4-4e44-bb0e-96fcb423608d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:25.350760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:25.381497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 129/243] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:25 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:25.463664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723725e3-1cb4-4e44-bb0e-96fcb423608d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:25.503213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5086e72-66ec-413e-9e95-89806767a5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:25.504121 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5086e72-66ec-413e-9e95-89806767a5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.504574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5086e72-66ec-413e-9e95-89806767a5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.505017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5086e72-66ec-413e-9e95-89806767a5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:25.505993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5086e72-66ec-413e-9e95-89806767a5b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:25.506471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5086e72-66ec-413e-9e95-89806767a5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:25.508119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5086e72-66ec-413e-9e95-89806767a5b5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:25.518591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 115/244] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:25.525920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd1227-fa23-436a-b648-9758239c6818 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:25.661522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:25.678067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd1227-fa23-436a-b648-9758239c6818 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:25.701859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5086e72-66ec-413e-9e95-89806767a5b5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:25.726748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c132f8-3f1e-4a56-be0b-4d51968534dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:25.728064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c132f8-3f1e-4a56-be0b-4d51968534dc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.728547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c132f8-3f1e-4a56-be0b-4d51968534dc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.728996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c132f8-3f1e-4a56-be0b-4d51968534dc None admin] PATH_INFO: `/v3/users/d2371d02ad9f40d8ad16ff498620df9d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:25.749341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c132f8-3f1e-4a56-be0b-4d51968534dc None admin] RBAC: Authorizing `identity:get_user(user_id=d2371d02ad9f40d8ad16ff498620df9d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:25.750693 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c132f8-3f1e-4a56-be0b-4d51968534dc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:25.752976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 130/245] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:25 2022] GET /identity/v3/users/d2371d02ad9f40d8ad16ff498620df9d => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:25.770345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c132f8-3f1e-4a56-be0b-4d51968534dc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:25.816096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:25.882143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5086e72-66ec-413e-9e95-89806767a5b5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:25.934328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c132f8-3f1e-4a56-be0b-4d51968534dc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:25.938658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b081da-b25d-4af6-8b52-bf9e40ded8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:25.938809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b081da-b25d-4af6-8b52-bf9e40ded8fd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.939032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b081da-b25d-4af6-8b52-bf9e40ded8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.939308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b081da-b25d-4af6-8b52-bf9e40ded8fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:25.954691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b081da-b25d-4af6-8b52-bf9e40ded8fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-87b081da-b25d-4af6-8b52-bf9e40ded8fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:25.960442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:25.962048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 116/246] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:04:25.962850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d27ddd-4c05-4451-9810-732ddcb33304 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.963555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d27ddd-4c05-4451-9810-732ddcb33304 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.963555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d27ddd-4c05-4451-9810-732ddcb33304 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:25.966117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 117/247] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:25.971798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d27ddd-4c05-4451-9810-732ddcb33304 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:25.982991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd07803-a395-4ae3-a434-4d9fc33b32f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:25.983607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd07803-a395-4ae3-a434-4d9fc33b32f0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:25.983882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd07803-a395-4ae3-a434-4d9fc33b32f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:25.984150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd07803-a395-4ae3-a434-4d9fc33b32f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:26.004428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:26.084217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd07803-a395-4ae3-a434-4d9fc33b32f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:26.086332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd07803-a395-4ae3-a434-4d9fc33b32f0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:26.112566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d27ddd-4c05-4451-9810-732ddcb33304 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:26.163876 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:26.177260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38ee27-c2f2-44f0-9dfb-e25bfe4caf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:26.178779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38ee27-c2f2-44f0-9dfb-e25bfe4caf95 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:26.179261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38ee27-c2f2-44f0-9dfb-e25bfe4caf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:26.179729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38ee27-c2f2-44f0-9dfb-e25bfe4caf95 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:26.180744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38ee27-c2f2-44f0-9dfb-e25bfe4caf95 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:26.181241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38ee27-c2f2-44f0-9dfb-e25bfe4caf95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:26.182871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38ee27-c2f2-44f0-9dfb-e25bfe4caf95 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:26.200059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 118/248] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:26.200059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22538fae-90e6-422a-931d-a76fd3803081 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:26.200059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22538fae-90e6-422a-931d-a76fd3803081 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:26.200059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22538fae-90e6-422a-931d-a76fd3803081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:26.234052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-22538fae-90e6-422a-931d-a76fd3803081 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:26.235263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-22538fae-90e6-422a-931d-a76fd3803081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:26.365192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 131/249] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:26.366095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd07803-a395-4ae3-a434-4d9fc33b32f0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:26.399630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 119/250] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:26.400542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22538fae-90e6-422a-931d-a76fd3803081 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:26.443152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:26.565861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd07803-a395-4ae3-a434-4d9fc33b32f0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:26.573857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22538fae-90e6-422a-931d-a76fd3803081 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:26.598742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:26.620442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe26ae-cb42-4195-9d03-57c6e3874176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:26.621541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe26ae-cb42-4195-9d03-57c6e3874176 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:26.622142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe26ae-cb42-4195-9d03-57c6e3874176 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:26.623234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe26ae-cb42-4195-9d03-57c6e3874176 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:26.624638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe26ae-cb42-4195-9d03-57c6e3874176 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:26.627497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe26ae-cb42-4195-9d03-57c6e3874176 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:26.629826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 120/251] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:26 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:26.637815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f084184-d96e-427d-92e7-0899993d1a3a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:26.638459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f084184-d96e-427d-92e7-0899993d1a3a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:26.639018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f084184-d96e-427d-92e7-0899993d1a3a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:26.642175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 121/252] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:26.643329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f084184-d96e-427d-92e7-0899993d1a3a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:26.670297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a7e0b-0cdf-485a-803e-43d9669ad2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:26.670950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a7e0b-0cdf-485a-803e-43d9669ad2cc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:26.671214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a7e0b-0cdf-485a-803e-43d9669ad2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:26.671487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a7e0b-0cdf-485a-803e-43d9669ad2cc None admin] PATH_INFO: `/v3/users/e2f1b018728f4931a9ee0dbdf0917789` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:26.673354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a7e0b-0cdf-485a-803e-43d9669ad2cc None admin] RBAC: Authorizing `identity:get_user(user_id=e2f1b018728f4931a9ee0dbdf0917789)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:26.676687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a7e0b-0cdf-485a-803e-43d9669ad2cc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:26.678473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 132/253] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:26 2022] GET /identity/v3/users/e2f1b018728f4931a9ee0dbdf0917789 => generated 297 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:26.681536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2f89a-d36e-4e32-92fb-779adfbaf9b4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:26.681811 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2f89a-d36e-4e32-92fb-779adfbaf9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:26.682108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2f89a-d36e-4e32-92fb-779adfbaf9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:26.726053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1cd2f89a-d36e-4e32-92fb-779adfbaf9b4 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:26.727600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1cd2f89a-d36e-4e32-92fb-779adfbaf9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:26.806432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f084184-d96e-427d-92e7-0899993d1a3a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:26.829948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 133/254] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:26.829948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2f89a-d36e-4e32-92fb-779adfbaf9b4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:26.869407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c102e-d14c-4716-ab04-5cbc65024c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:26.870630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c102e-d14c-4716-ab04-5cbc65024c3b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:26.871090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c102e-d14c-4716-ab04-5cbc65024c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:26.871519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c102e-d14c-4716-ab04-5cbc65024c3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:26.872540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c102e-d14c-4716-ab04-5cbc65024c3b None admin] RBAC: Adding query filter params (role.id=0f2f4356fa294a7db98c5636fe2d390d, scope.project.id=858cb936469e4382b567530855fa7507, user.id=d2371d02ad9f40d8ad16ff498620df9d) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:26.873914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c102e-d14c-4716-ab04-5cbc65024c3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:26.876299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c102e-d14c-4716-ab04-5cbc65024c3b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:26.890410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 122/255] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:26 2022] GET /identity/v3/role_assignments?user.id=d2371d02ad9f40d8ad16ff498620df9d&scope.project.id=858cb936469e4382b567530855fa7507&role.id=0f2f4356fa294a7db98c5636fe2d390d => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:26.891729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c102e-d14c-4716-ab04-5cbc65024c3b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:26.918460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:27.004030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:27.035352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2f89a-d36e-4e32-92fb-779adfbaf9b4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:27.110142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c102e-d14c-4716-ab04-5cbc65024c3b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:27.128884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae15bb7-1d55-4958-8860-b4fbf1324924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:27.129457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae15bb7-1d55-4958-8860-b4fbf1324924 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:27.129737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae15bb7-1d55-4958-8860-b4fbf1324924 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:27.130115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae15bb7-1d55-4958-8860-b4fbf1324924 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:27.132675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae15bb7-1d55-4958-8860-b4fbf1324924 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:27.134175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae15bb7-1d55-4958-8860-b4fbf1324924 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:27.138850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 134/256] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:26 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:27.148930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae15bb7-1d55-4958-8860-b4fbf1324924 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:27.202662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb257d-7c95-47c7-ad69-bb2d6493bb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:27.203323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb257d-7c95-47c7-ad69-bb2d6493bb14 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:27.203605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb257d-7c95-47c7-ad69-bb2d6493bb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:27.203879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb257d-7c95-47c7-ad69-bb2d6493bb14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:27.238256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb257d-7c95-47c7-ad69-bb2d6493bb14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-01fb257d-7c95-47c7-ad69-bb2d6493bb14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:27.243739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:27.244686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:27.246308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 123/257] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:27.251003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb257d-7c95-47c7-ad69-bb2d6493bb14 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:27.315298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae15bb7-1d55-4958-8860-b4fbf1324924 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:27.331822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:27.355909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b782a-c4a7-4773-9af4-5c085764bc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:27.356590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b782a-c4a7-4773-9af4-5c085764bc56 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:27.356926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b782a-c4a7-4773-9af4-5c085764bc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:27.357202 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b782a-c4a7-4773-9af4-5c085764bc56 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507/users/e2f1b018728f4931a9ee0dbdf0917789/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:27.359639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b782a-c4a7-4773-9af4-5c085764bc56 None admin] RBAC: Authorizing `identity:create_grant(project_id=858cb936469e4382b567530855fa7507, user_id=e2f1b018728f4931a9ee0dbdf0917789, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:27.361114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b782a-c4a7-4773-9af4-5c085764bc56 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:27.379349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb257d-7c95-47c7-ad69-bb2d6493bb14 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:27.390449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:27.416298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca0b5d-aea7-4d78-a08d-a9cdf9a67ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:27.416919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca0b5d-aea7-4d78-a08d-a9cdf9a67ddc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:27.417197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca0b5d-aea7-4d78-a08d-a9cdf9a67ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:27.428681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca0b5d-aea7-4d78-a08d-a9cdf9a67ddc None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:27.428681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca0b5d-aea7-4d78-a08d-a9cdf9a67ddc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:27.428681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca0b5d-aea7-4d78-a08d-a9cdf9a67ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:27.428681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca0b5d-aea7-4d78-a08d-a9cdf9a67ddc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:27.430752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 124/258] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:27.431621 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca0b5d-aea7-4d78-a08d-a9cdf9a67ddc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:27.464475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 135/259] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:04:27 2022] PUT /identity/v3/projects/858cb936469e4382b567530855fa7507/users/e2f1b018728f4931a9ee0dbdf0917789/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:04:27.465342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b782a-c4a7-4773-9af4-5c085764bc56 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:27.546825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca0b5d-aea7-4d78-a08d-a9cdf9a67ddc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:27.589003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b787488-8dcc-4555-8105-2596d529a9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:27.589657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b787488-8dcc-4555-8105-2596d529a9cb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:27.589966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b787488-8dcc-4555-8105-2596d529a9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:27.590241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b787488-8dcc-4555-8105-2596d529a9cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:27.602535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b787488-8dcc-4555-8105-2596d529a9cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-5b787488-8dcc-4555-8105-2596d529a9cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:27.607647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:27.608673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:27.609470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 125/260] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:27.615169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b787488-8dcc-4555-8105-2596d529a9cb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:27.650369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:27.722694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b782a-c4a7-4773-9af4-5c085764bc56 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:27.771429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b787488-8dcc-4555-8105-2596d529a9cb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:27.794072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570ca96-8fbb-4b2a-baa4-8926d52c6362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:27.794728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570ca96-8fbb-4b2a-baa4-8926d52c6362 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:27.795008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570ca96-8fbb-4b2a-baa4-8926d52c6362 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:27.795281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570ca96-8fbb-4b2a-baa4-8926d52c6362 None admin] PATH_INFO: `/v3/users/ad316a97ff1a405281301cf0f697665f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:27.796455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570ca96-8fbb-4b2a-baa4-8926d52c6362 None admin] RBAC: Authorizing `identity:get_user(user_id=ad316a97ff1a405281301cf0f697665f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:27.799473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570ca96-8fbb-4b2a-baa4-8926d52c6362 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:27.801472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 136/261] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:27 2022] GET /identity/v3/users/ad316a97ff1a405281301cf0f697665f => generated 283 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:27.806957 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570ca96-8fbb-4b2a-baa4-8926d52c6362 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:27.857931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:27.874613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e9f58-6eb5-4157-a949-41d7e12cadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:27.875375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e9f58-6eb5-4157-a949-41d7e12cadd1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:27.875692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e9f58-6eb5-4157-a949-41d7e12cadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:27.875992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e9f58-6eb5-4157-a949-41d7e12cadd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:27.876763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e9f58-6eb5-4157-a949-41d7e12cadd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:27.877108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e9f58-6eb5-4157-a949-41d7e12cadd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:27.878560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e9f58-6eb5-4157-a949-41d7e12cadd1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:27.893584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 126/262] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:27.916672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e9f58-6eb5-4157-a949-41d7e12cadd1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:27.935420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570ca96-8fbb-4b2a-baa4-8926d52c6362 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:27.978036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a33e9-4594-4e8c-93a7-16fb33b10e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:27.979247 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a33e9-4594-4e8c-93a7-16fb33b10e07 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:27.979704 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a33e9-4594-4e8c-93a7-16fb33b10e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:27.980050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a33e9-4594-4e8c-93a7-16fb33b10e07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:28.095177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a33e9-4594-4e8c-93a7-16fb33b10e07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:28.098933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:28.101469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a33e9-4594-4e8c-93a7-16fb33b10e07 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:28.115017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e9f58-6eb5-4157-a949-41d7e12cadd1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:28.177890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7e485-c80b-4f68-a693-c6c938d577f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:28.178714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e485-c80b-4f68-a693-c6c938d577f8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:28.179257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e485-c80b-4f68-a693-c6c938d577f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:28.180162 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e485-c80b-4f68-a693-c6c938d577f8 None admin] PATH_INFO: `/v3/users/ad316a97ff1a405281301cf0f697665f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:28.182215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7e485-c80b-4f68-a693-c6c938d577f8 None admin] RBAC: Authorizing `identity:get_user(user_id=ad316a97ff1a405281301cf0f697665f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:28.185395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7e485-c80b-4f68-a693-c6c938d577f8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:28.188902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 127/263] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:27 2022] GET /identity/v3/users/ad316a97ff1a405281301cf0f697665f => generated 283 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:28.197142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:28.217707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7e485-c80b-4f68-a693-c6c938d577f8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:28.333907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 137/264] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:04:28.335735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a33e9-4594-4e8c-93a7-16fb33b10e07 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:28.540117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a33e9-4594-4e8c-93a7-16fb33b10e07 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:28.577586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7e485-c80b-4f68-a693-c6c938d577f8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:28.610040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdeb45-6359-442f-bfa0-94c18558f8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:28.611119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdeb45-6359-442f-bfa0-94c18558f8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:28.612169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdeb45-6359-442f-bfa0-94c18558f8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:28.612647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdeb45-6359-442f-bfa0-94c18558f8d0 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:28.614753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdeb45-6359-442f-bfa0-94c18558f8d0 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:28.618171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdeb45-6359-442f-bfa0-94c18558f8d0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:28.621729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 138/265] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:28 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:28.632661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdeb45-6359-442f-bfa0-94c18558f8d0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:28.654277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:28.698328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d7a1c-4e56-45b6-ba89-f11311a42a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:28.698958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7a1c-4e56-45b6-ba89-f11311a42a46 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:28.699231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7a1c-4e56-45b6-ba89-f11311a42a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:28.699498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7a1c-4e56-45b6-ba89-f11311a42a46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:28.752199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:28.802298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d7a1c-4e56-45b6-ba89-f11311a42a46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:28.803803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d7a1c-4e56-45b6-ba89-f11311a42a46 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:28.811280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdeb45-6359-442f-bfa0-94c18558f8d0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:28.855084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32259e2-9f8b-4470-a45b-0baa5cb1ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:28.855703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32259e2-9f8b-4470-a45b-0baa5cb1ab29 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:28.855984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32259e2-9f8b-4470-a45b-0baa5cb1ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:28.856260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32259e2-9f8b-4470-a45b-0baa5cb1ab29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:28.857361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32259e2-9f8b-4470-a45b-0baa5cb1ab29 None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=858cb936469e4382b567530855fa7507, user.id=ad316a97ff1a405281301cf0f697665f) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:28.857675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32259e2-9f8b-4470-a45b-0baa5cb1ab29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:28.859529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32259e2-9f8b-4470-a45b-0baa5cb1ab29 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:28.868185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 139/266] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:28 2022] GET /identity/v3/role_assignments?user.id=ad316a97ff1a405281301cf0f697665f&scope.project.id=858cb936469e4382b567530855fa7507&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:28.869269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af398-af58-4b5a-b8c4-c2da09125645 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:28.869552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af398-af58-4b5a-b8c4-c2da09125645 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:28.878172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af398-af58-4b5a-b8c4-c2da09125645 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:28.879975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 140/267] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:28.881253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584d31-5592-4799-b1c0-33b586cb9083 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:28.881530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584d31-5592-4799-b1c0-33b586cb9083 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:28.881802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584d31-5592-4799-b1c0-33b586cb9083 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:28.883065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 141/268] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:28.886628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573af0d3-9d17-49ad-acdf-b8c3fcc43d80 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:28.886911 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573af0d3-9d17-49ad-acdf-b8c3fcc43d80 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:28.887162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573af0d3-9d17-49ad-acdf-b8c3fcc43d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:28.923078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-573af0d3-9d17-49ad-acdf-b8c3fcc43d80 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:28.924903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-573af0d3-9d17-49ad-acdf-b8c3fcc43d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:28.939490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 128/269] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:04:28.940827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0af463-de69-48ac-a727-13bfdf542fd5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:28.941109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0af463-de69-48ac-a727-13bfdf542fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:28.941385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0af463-de69-48ac-a727-13bfdf542fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:28.977052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fd0af463-de69-48ac-a727-13bfdf542fd5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:28.978419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fd0af463-de69-48ac-a727-13bfdf542fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:29.022559 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 142/270] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:29.024048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573af0d3-9d17-49ad-acdf-b8c3fcc43d80 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:29.029137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:29.079873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:29.131692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 129/271] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:29.161364 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573af0d3-9d17-49ad-acdf-b8c3fcc43d80 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:29.254504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3f7e0-342a-47d6-a891-f0652ec180be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:29.255154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3f7e0-342a-47d6-a891-f0652ec180be None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:29.255434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3f7e0-342a-47d6-a891-f0652ec180be None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:29.255707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3f7e0-342a-47d6-a891-f0652ec180be None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:29.256876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3f7e0-342a-47d6-a891-f0652ec180be None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:29.258377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3f7e0-342a-47d6-a891-f0652ec180be None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:29.260428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 143/272] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:29 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:29.266479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0af463-de69-48ac-a727-13bfdf542fd5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:29.414377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3f7e0-342a-47d6-a891-f0652ec180be None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:29.453931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0af463-de69-48ac-a727-13bfdf542fd5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:29.514719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:29.537366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24817f6-75e1-4db1-b6bb-f6b34fe205d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:29.538027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24817f6-75e1-4db1-b6bb-f6b34fe205d9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:29.538374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24817f6-75e1-4db1-b6bb-f6b34fe205d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:29.538651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24817f6-75e1-4db1-b6bb-f6b34fe205d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:29.539619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24817f6-75e1-4db1-b6bb-f6b34fe205d9 None admin] RBAC: Adding query filter params (role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=858cb936469e4382b567530855fa7507, user.id=ad316a97ff1a405281301cf0f697665f) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:29.539956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24817f6-75e1-4db1-b6bb-f6b34fe205d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:29.542975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24817f6-75e1-4db1-b6bb-f6b34fe205d9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:29.545140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3f7e0-342a-47d6-a891-f0652ec180be None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:29.552662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 130/273] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:29 2022] GET /identity/v3/role_assignments?user.id=ad316a97ff1a405281301cf0f697665f&scope.project.id=858cb936469e4382b567530855fa7507&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:29.553681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24817f6-75e1-4db1-b6bb-f6b34fe205d9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:29.618890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f867bd-3f9c-433e-a6c4-60fabf6d9614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:29.619571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f867bd-3f9c-433e-a6c4-60fabf6d9614 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:29.619852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f867bd-3f9c-433e-a6c4-60fabf6d9614 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:29.620140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f867bd-3f9c-433e-a6c4-60fabf6d9614 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:29.621482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f867bd-3f9c-433e-a6c4-60fabf6d9614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:29.623238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f867bd-3f9c-433e-a6c4-60fabf6d9614 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:29.625068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 144/274] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:29.632015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f867bd-3f9c-433e-a6c4-60fabf6d9614 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:29.661694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:29.716300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24817f6-75e1-4db1-b6bb-f6b34fe205d9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:29.756012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171369f-6133-42ee-9c43-f4504ebeba07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:29.756597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171369f-6133-42ee-9c43-f4504ebeba07 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:29.757032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171369f-6133-42ee-9c43-f4504ebeba07 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:29.757393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171369f-6133-42ee-9c43-f4504ebeba07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:29.770574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171369f-6133-42ee-9c43-f4504ebeba07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a171369f-6133-42ee-9c43-f4504ebeba07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:29.775997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:29.776908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:29.777778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 131/275] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:29.778507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5a682-ce22-418e-951e-23349767da7c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:29.778761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5a682-ce22-418e-951e-23349767da7c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:29.779174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5a682-ce22-418e-951e-23349767da7c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:29.780739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 132/276] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:29.783897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5a682-ce22-418e-951e-23349767da7c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:29.789259 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f867bd-3f9c-433e-a6c4-60fabf6d9614 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:29.840868 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a858-ccb3-48d7-b9f6-8001768c0500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:29.841751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a858-ccb3-48d7-b9f6-8001768c0500 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:29.842068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a858-ccb3-48d7-b9f6-8001768c0500 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:29.842825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a858-ccb3-48d7-b9f6-8001768c0500 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:29.843827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a858-ccb3-48d7-b9f6-8001768c0500 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:29.845624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a858-ccb3-48d7-b9f6-8001768c0500 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:29.859557 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:29.902472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5a682-ce22-418e-951e-23349767da7c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:29.918113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:29.941781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a6919-1564-4b66-be1d-46d46577798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:29.942420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a6919-1564-4b66-be1d-46d46577798c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:29.942698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a6919-1564-4b66-be1d-46d46577798c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:29.942972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a6919-1564-4b66-be1d-46d46577798c None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:29.943642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a6919-1564-4b66-be1d-46d46577798c None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:29.943943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a6919-1564-4b66-be1d-46d46577798c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:29.945383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a6919-1564-4b66-be1d-46d46577798c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:29.953879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 133/277] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:29.955180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f0a86-eefc-461e-8668-dbcba5e20268 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:29.955475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f0a86-eefc-461e-8668-dbcba5e20268 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:29.955739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f0a86-eefc-461e-8668-dbcba5e20268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:29.975297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e08f0a86-eefc-461e-8668-dbcba5e20268 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:29.976403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e08f0a86-eefc-461e-8668-dbcba5e20268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:30.017919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 145/278] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:29 2022] POST /identity/v3/users => generated 293 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:04:30.026135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a858-ccb3-48d7-b9f6-8001768c0500 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:30.056606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 134/279] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:30.159228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a858-ccb3-48d7-b9f6-8001768c0500 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:30.201034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d142bce-d455-4fc6-a7a5-f228194c581c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:30.201781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142bce-d455-4fc6-a7a5-f228194c581c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:30.202089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142bce-d455-4fc6-a7a5-f228194c581c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:30.202414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142bce-d455-4fc6-a7a5-f228194c581c None admin] PATH_INFO: `/v3/users/7535b31484054f789e4df1d330e3491b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:30.218745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d142bce-d455-4fc6-a7a5-f228194c581c None admin] RBAC: Authorizing `identity:get_user(user_id=7535b31484054f789e4df1d330e3491b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:30.220351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d142bce-d455-4fc6-a7a5-f228194c581c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:30.222299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 146/280] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:30 2022] GET /identity/v3/users/7535b31484054f789e4df1d330e3491b => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:30.228852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f0a86-eefc-461e-8668-dbcba5e20268 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:30.335100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d142bce-d455-4fc6-a7a5-f228194c581c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:30.367589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f0a86-eefc-461e-8668-dbcba5e20268 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:30.376168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:30.407798 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:30.413072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f57e9f-fe2e-49d5-8452-614c1687528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:30.413698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f57e9f-fe2e-49d5-8452-614c1687528f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:30.414016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f57e9f-fe2e-49d5-8452-614c1687528f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:30.414295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f57e9f-fe2e-49d5-8452-614c1687528f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:30.415042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f57e9f-fe2e-49d5-8452-614c1687528f None admin] RBAC: Adding query filter params (role.id=799b2f3554434281bf10b2602fbbc09a, user.id=7535b31484054f789e4df1d330e3491b, scope.system=all) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:30.415349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f57e9f-fe2e-49d5-8452-614c1687528f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:30.416858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f57e9f-fe2e-49d5-8452-614c1687528f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:30.422788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 135/281] 10.222.0.83 () {56 vars in 1308 bytes} [Wed Aug 24 11:04:30 2022] GET /identity/v3/role_assignments?user.id=7535b31484054f789e4df1d330e3491b&scope.system=all&role.id=799b2f3554434281bf10b2602fbbc09a => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:30.424246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908b6d-a06e-45df-853c-532c933cb920 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:30.424510 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908b6d-a06e-45df-853c-532c933cb920 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:30.424823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908b6d-a06e-45df-853c-532c933cb920 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:30.426477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 136/282] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:30.434439 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a83b0b-cba7-4fbf-99bc-1f2f880f3e2c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:30.434764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a83b0b-cba7-4fbf-99bc-1f2f880f3e2c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:30.435525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a83b0b-cba7-4fbf-99bc-1f2f880f3e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:30.462769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-18a83b0b-cba7-4fbf-99bc-1f2f880f3e2c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:30.464024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-18a83b0b-cba7-4fbf-99bc-1f2f880f3e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:30.478461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d142bce-d455-4fc6-a7a5-f228194c581c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:30.524245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831400b-b9ce-4cec-99a6-13863a098343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:30.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831400b-b9ce-4cec-99a6-13863a098343 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:30.525098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831400b-b9ce-4cec-99a6-13863a098343 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:30.525373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831400b-b9ce-4cec-99a6-13863a098343 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:30.541437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831400b-b9ce-4cec-99a6-13863a098343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e831400b-b9ce-4cec-99a6-13863a098343 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:30.546770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:30.548128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:30.548965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 147/283] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:04:30.556268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831400b-b9ce-4cec-99a6-13863a098343 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:30.557270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 137/284] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:30.702800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831400b-b9ce-4cec-99a6-13863a098343 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:30.725180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:30.759048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e21b7-93ed-41db-ae85-8ee033e4658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:30.759669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e21b7-93ed-41db-ae85-8ee033e4658f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:30.759948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e21b7-93ed-41db-ae85-8ee033e4658f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:30.760220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e21b7-93ed-41db-ae85-8ee033e4658f None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:30.760906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e21b7-93ed-41db-ae85-8ee033e4658f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:30.761211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e21b7-93ed-41db-ae85-8ee033e4658f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:30.762659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e21b7-93ed-41db-ae85-8ee033e4658f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:30.779385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 148/285] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:30.785722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a83b0b-cba7-4fbf-99bc-1f2f880f3e2c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:30.910966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a83b0b-cba7-4fbf-99bc-1f2f880f3e2c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:30.914229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:30.948936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa33e52-b53a-46d8-bd77-2f9e6af4094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:30.949577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa33e52-b53a-46d8-bd77-2f9e6af4094d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:30.949854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa33e52-b53a-46d8-bd77-2f9e6af4094d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:30.950152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa33e52-b53a-46d8-bd77-2f9e6af4094d None admin] PATH_INFO: `/v3/users/d2371d02ad9f40d8ad16ff498620df9d` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:30.951443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa33e52-b53a-46d8-bd77-2f9e6af4094d None admin] RBAC: Authorizing `identity:get_user(user_id=d2371d02ad9f40d8ad16ff498620df9d)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:30.952962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa33e52-b53a-46d8-bd77-2f9e6af4094d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:30.955634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 138/286] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:30 2022] GET /identity/v3/users/d2371d02ad9f40d8ad16ff498620df9d => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:30.961190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e21b7-93ed-41db-ae85-8ee033e4658f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:31.000031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa33e52-b53a-46d8-bd77-2f9e6af4094d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:31.152575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa33e52-b53a-46d8-bd77-2f9e6af4094d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:31.174322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e21b7-93ed-41db-ae85-8ee033e4658f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:31.197929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59018f19-0dad-4865-952e-d94c1769036e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:31.198577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59018f19-0dad-4865-952e-d94c1769036e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:31.198856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59018f19-0dad-4865-952e-d94c1769036e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:31.199126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59018f19-0dad-4865-952e-d94c1769036e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:31.219218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59018f19-0dad-4865-952e-d94c1769036e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-59018f19-0dad-4865-952e-d94c1769036e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:31.224092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:31.226207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:31.228110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 139/287] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:04:31.237900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59018f19-0dad-4865-952e-d94c1769036e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:31.242783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c06276-228b-4a2e-9d95-65b2855e5118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:31.242783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c06276-228b-4a2e-9d95-65b2855e5118 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:31.243206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c06276-228b-4a2e-9d95-65b2855e5118 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:31.243206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c06276-228b-4a2e-9d95-65b2855e5118 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:31.244324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c06276-228b-4a2e-9d95-65b2855e5118 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:31.245791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c06276-228b-4a2e-9d95-65b2855e5118 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:31.247558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 149/288] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:30 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:31.253297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c06276-228b-4a2e-9d95-65b2855e5118 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:31.294239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:31.303308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:31.413797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59018f19-0dad-4865-952e-d94c1769036e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:31.418749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c06276-228b-4a2e-9d95-65b2855e5118 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:31.497750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dbc9d-dee8-4824-af01-be35adbfa631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:31.498765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dbc9d-dee8-4824-af01-be35adbfa631 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:31.499181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dbc9d-dee8-4824-af01-be35adbfa631 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:31.499591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dbc9d-dee8-4824-af01-be35adbfa631 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507/users/d2371d02ad9f40d8ad16ff498620df9d/roles/0f2f4356fa294a7db98c5636fe2d390d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:31.504517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5dbc9d-dee8-4824-af01-be35adbfa631 None admin] RBAC: Authorizing `identity:create_grant(project_id=858cb936469e4382b567530855fa7507, user_id=d2371d02ad9f40d8ad16ff498620df9d, role_id=0f2f4356fa294a7db98c5636fe2d390d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:31.506059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5dbc9d-dee8-4824-af01-be35adbfa631 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:31.508988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a8a0e-72a7-44f4-964e-b527231a5d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:31.509901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a8a0e-72a7-44f4-964e-b527231a5d33 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:31.510330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a8a0e-72a7-44f4-964e-b527231a5d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:31.510734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a8a0e-72a7-44f4-964e-b527231a5d33 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:31.511802 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a8a0e-72a7-44f4-964e-b527231a5d33 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:31.512371 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a8a0e-72a7-44f4-964e-b527231a5d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:31.514183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a8a0e-72a7-44f4-964e-b527231a5d33 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:31.527890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 140/289] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:31.534751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a8a0e-72a7-44f4-964e-b527231a5d33 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:31.599722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 150/290] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:04:31 2022] PUT /identity/v3/projects/858cb936469e4382b567530855fa7507/users/d2371d02ad9f40d8ad16ff498620df9d/roles/0f2f4356fa294a7db98c5636fe2d390d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:31.640092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:31.652555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a8a0e-72a7-44f4-964e-b527231a5d33 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:31.692909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaa78a-e9ca-4ef4-92da-e70b6ced71ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:31.693474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaa78a-e9ca-4ef4-92da-e70b6ced71ad None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:31.694040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaa78a-e9ca-4ef4-92da-e70b6ced71ad None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:31.694811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaa78a-e9ca-4ef4-92da-e70b6ced71ad None admin] PATH_INFO: `/v3/users/e2f1b018728f4931a9ee0dbdf0917789` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:31.697234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbaa78a-e9ca-4ef4-92da-e70b6ced71ad None admin] RBAC: Authorizing `identity:get_user(user_id=e2f1b018728f4931a9ee0dbdf0917789)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:31.699122 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbaa78a-e9ca-4ef4-92da-e70b6ced71ad None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:31.700891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 141/291] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:31 2022] GET /identity/v3/users/e2f1b018728f4931a9ee0dbdf0917789 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:31.707785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dbc9d-dee8-4824-af01-be35adbfa631 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:31.846603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dbc9d-dee8-4824-af01-be35adbfa631 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:31.848483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:31.886287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4d5c-01e1-46ff-b435-77d20f1f9ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:31.886951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4d5c-01e1-46ff-b435-77d20f1f9ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:31.887233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4d5c-01e1-46ff-b435-77d20f1f9ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:31.887509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4d5c-01e1-46ff-b435-77d20f1f9ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:31.968418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c4d5c-01e1-46ff-b435-77d20f1f9ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:31.969831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c4d5c-01e1-46ff-b435-77d20f1f9ac5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:32.077804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 151/292] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:32.084100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaa78a-e9ca-4ef4-92da-e70b6ced71ad None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:32.230467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaa78a-e9ca-4ef4-92da-e70b6ced71ad None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:32.265863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b68c61-5bc3-4b00-96f7-2f1d8726b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:32.266529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b68c61-5bc3-4b00-96f7-2f1d8726b226 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:32.266811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b68c61-5bc3-4b00-96f7-2f1d8726b226 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:32.267088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b68c61-5bc3-4b00-96f7-2f1d8726b226 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:32.268784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b68c61-5bc3-4b00-96f7-2f1d8726b226 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:32.270278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b68c61-5bc3-4b00-96f7-2f1d8726b226 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:32.272287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 142/293] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:32 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:32.278196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4d5c-01e1-46ff-b435-77d20f1f9ac5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:32.280423 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:32.449696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4d5c-01e1-46ff-b435-77d20f1f9ac5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:32.497118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05913f51-ac43-409d-8ec2-a76756b6c7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:32.498099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05913f51-ac43-409d-8ec2-a76756b6c7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:32.498580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05913f51-ac43-409d-8ec2-a76756b6c7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:32.499069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05913f51-ac43-409d-8ec2-a76756b6c7a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:32.500027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05913f51-ac43-409d-8ec2-a76756b6c7a4 None admin] RBAC: Adding query filter params (role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=858cb936469e4382b567530855fa7507, user.id=e2f1b018728f4931a9ee0dbdf0917789) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:32.500534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05913f51-ac43-409d-8ec2-a76756b6c7a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:32.502785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05913f51-ac43-409d-8ec2-a76756b6c7a4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:32.514610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 152/294] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:32 2022] GET /identity/v3/role_assignments?user.id=e2f1b018728f4931a9ee0dbdf0917789&scope.project.id=858cb936469e4382b567530855fa7507&role.id=799b2f3554434281bf10b2602fbbc09a => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:32.662231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cda881-0ee5-48b6-ba8e-3dc2447f292d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:32.662612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cda881-0ee5-48b6-ba8e-3dc2447f292d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:32.662788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cda881-0ee5-48b6-ba8e-3dc2447f292d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:32.664549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 143/295] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:32.670440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0e318-c80b-4433-8f32-645076356e6e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:32.671362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0e318-c80b-4433-8f32-645076356e6e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:32.671888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0e318-c80b-4433-8f32-645076356e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:32.699770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b3b0e318-c80b-4433-8f32-645076356e6e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:32.701354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b3b0e318-c80b-4433-8f32-645076356e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:32.716561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab2fcb-8453-4177-94e7-75edbd5522e7 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:32.716850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab2fcb-8453-4177-94e7-75edbd5522e7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:32.717384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab2fcb-8453-4177-94e7-75edbd5522e7 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:32.719441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 144/296] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:32.725959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8fe34-f4e2-4626-8999-4629f19e2444 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:32.726573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8fe34-f4e2-4626-8999-4629f19e2444 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:32.727125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8fe34-f4e2-4626-8999-4629f19e2444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:32.748347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2db8fe34-f4e2-4626-8999-4629f19e2444 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:32.749886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2db8fe34-f4e2-4626-8999-4629f19e2444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:32.773096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:32.810903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 153/297] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:32.837588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 145/298] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:32.996180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09363b61-1bca-4459-9977-b667a22e49fe None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:32.996476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09363b61-1bca-4459-9977-b667a22e49fe None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:32.997211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09363b61-1bca-4459-9977-b667a22e49fe None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:32.999706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 154/299] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:33.005958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd825a-e2bb-422a-9a85-c21d8f817d5d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:33.006245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd825a-e2bb-422a-9a85-c21d8f817d5d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:33.006722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd825a-e2bb-422a-9a85-c21d8f817d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:33.027147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-08dd825a-e2bb-422a-9a85-c21d8f817d5d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:33.028705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-08dd825a-e2bb-422a-9a85-c21d8f817d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:33.120153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09363b61-1bca-4459-9977-b667a22e49fe None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:33.129946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 146/300] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:33.140585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd825a-e2bb-422a-9a85-c21d8f817d5d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:33.259774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:33.276727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09363b61-1bca-4459-9977-b667a22e49fe None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:33.279999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd825a-e2bb-422a-9a85-c21d8f817d5d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:33.328708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb43f9-c01d-42eb-91bf-d53bbf8fce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:33.329751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb43f9-c01d-42eb-91bf-d53bbf8fce93 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:33.330493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb43f9-c01d-42eb-91bf-d53bbf8fce93 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:33.330918 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb43f9-c01d-42eb-91bf-d53bbf8fce93 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:33.337349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22caccf-5832-49c1-ac89-b44d365465dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:33.338014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22caccf-5832-49c1-ac89-b44d365465dd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:33.338297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22caccf-5832-49c1-ac89-b44d365465dd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:33.338572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22caccf-5832-49c1-ac89-b44d365465dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:33.340008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22caccf-5832-49c1-ac89-b44d365465dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:33.341947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22caccf-5832-49c1-ac89-b44d365465dd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:33.344131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb43f9-c01d-42eb-91bf-d53bbf8fce93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:33.348934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 147/301] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:33.349780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22caccf-5832-49c1-ac89-b44d365465dd None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a9bb43f9-c01d-42eb-91bf-d53bbf8fce93 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:33.350630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:33.351731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:33.352625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:33.355002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 155/302] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:33.364651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb43f9-c01d-42eb-91bf-d53bbf8fce93 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:33.443956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:33.469024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22caccf-5832-49c1-ac89-b44d365465dd None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:33.493356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb43f9-c01d-42eb-91bf-d53bbf8fce93 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:33.510785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84734-365f-4ccd-a6a1-3a3cbfeec736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:33.511308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84734-365f-4ccd-a6a1-3a3cbfeec736 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:33.511585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84734-365f-4ccd-a6a1-3a3cbfeec736 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:33.511852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84734-365f-4ccd-a6a1-3a3cbfeec736 None admin] PATH_INFO: `/v3/groups` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:33.512493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b84734-365f-4ccd-a6a1-3a3cbfeec736 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:33.513965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b84734-365f-4ccd-a6a1-3a3cbfeec736 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:33.531913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3289b-9d98-4407-b2e3-5a51353a6ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:33.532522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3289b-9d98-4407-b2e3-5a51353a6ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:33.532795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3289b-9d98-4407-b2e3-5a51353a6ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:33.535475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3289b-9d98-4407-b2e3-5a51353a6ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:33.535475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3289b-9d98-4407-b2e3-5a51353a6ee6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:33.535475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3289b-9d98-4407-b2e3-5a51353a6ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:33.536050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3289b-9d98-4407-b2e3-5a51353a6ee6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:33.545632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 156/303] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:33.547677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3289b-9d98-4407-b2e3-5a51353a6ee6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:33.643025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 148/304] 10.222.0.83 () {60 vars in 1134 bytes} [Wed Aug 24 11:04:33 2022] POST /identity/v3/groups => generated 229 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:33.643025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84734-365f-4ccd-a6a1-3a3cbfeec736 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:33.677213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:33.686808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3289b-9d98-4407-b2e3-5a51353a6ee6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:33.706864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:33.728380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e96dea-3be0-49ee-8e93-6cd18843f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:33.729010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e96dea-3be0-49ee-8e93-6cd18843f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:33.729340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e96dea-3be0-49ee-8e93-6cd18843f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:33.729617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e96dea-3be0-49ee-8e93-6cd18843f2f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:33.744149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e96dea-3be0-49ee-8e93-6cd18843f2f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-35e96dea-3be0-49ee-8e93-6cd18843f2f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:33.749085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:33.750480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:33.751292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 157/305] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:33.758514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e96dea-3be0-49ee-8e93-6cd18843f2f2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:33.805623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84734-365f-4ccd-a6a1-3a3cbfeec736 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:33.853031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c45e8-d2b2-4b8f-9cb2-33a65ad31654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:33.853746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c45e8-d2b2-4b8f-9cb2-33a65ad31654 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:33.854167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c45e8-d2b2-4b8f-9cb2-33a65ad31654 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:33.854441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c45e8-d2b2-4b8f-9cb2-33a65ad31654 None admin] PATH_INFO: `/v3/users/c7cc7f5026e244629e66bad847def866` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:33.879819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c45e8-d2b2-4b8f-9cb2-33a65ad31654 None admin] RBAC: Authorizing `identity:get_user(user_id=c7cc7f5026e244629e66bad847def866)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:33.882329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c45e8-d2b2-4b8f-9cb2-33a65ad31654 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:33.885377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 149/306] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:33 2022] GET /identity/v3/users/c7cc7f5026e244629e66bad847def866 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:33.894954 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c45e8-d2b2-4b8f-9cb2-33a65ad31654 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:33.902359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e96dea-3be0-49ee-8e93-6cd18843f2f2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:33.960341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82d0cf-de08-4ddb-8122-a77928184be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:33.960970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82d0cf-de08-4ddb-8122-a77928184be3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:33.961254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82d0cf-de08-4ddb-8122-a77928184be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:33.961527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82d0cf-de08-4ddb-8122-a77928184be3 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:33.962251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82d0cf-de08-4ddb-8122-a77928184be3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:33.962591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82d0cf-de08-4ddb-8122-a77928184be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:33.964114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82d0cf-de08-4ddb-8122-a77928184be3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:33.975318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 158/307] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:33.981268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82d0cf-de08-4ddb-8122-a77928184be3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:34.005461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:34.032238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:34.043330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c45e8-d2b2-4b8f-9cb2-33a65ad31654 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:34.083527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff820a-e846-4bf3-aea2-47675398fd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:34.084485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff820a-e846-4bf3-aea2-47675398fd9f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:34.084984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff820a-e846-4bf3-aea2-47675398fd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:34.085559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff820a-e846-4bf3-aea2-47675398fd9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:34.086592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff820a-e846-4bf3-aea2-47675398fd9f None admin] RBAC: Adding query filter params (role.id=0f2f4356fa294a7db98c5636fe2d390d, user.id=c7cc7f5026e244629e66bad847def866, scope.system=all) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:34.087105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff820a-e846-4bf3-aea2-47675398fd9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:34.088914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff820a-e846-4bf3-aea2-47675398fd9f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:34.099463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 150/308] 10.222.0.83 () {56 vars in 1308 bytes} [Wed Aug 24 11:04:33 2022] GET /identity/v3/role_assignments?user.id=c7cc7f5026e244629e66bad847def866&scope.system=all&role.id=0f2f4356fa294a7db98c5636fe2d390d => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:34.109968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82d0cf-de08-4ddb-8122-a77928184be3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:34.153407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50fdc0-4e20-4073-9d4f-400e7c08fa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:34.154105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50fdc0-4e20-4073-9d4f-400e7c08fa16 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:34.154391 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50fdc0-4e20-4073-9d4f-400e7c08fa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:34.154663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50fdc0-4e20-4073-9d4f-400e7c08fa16 None admin] PATH_INFO: `/v3/users/7535b31484054f789e4df1d330e3491b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:34.155904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50fdc0-4e20-4073-9d4f-400e7c08fa16 None admin] RBAC: Authorizing `identity:get_user(user_id=7535b31484054f789e4df1d330e3491b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:34.156814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c762b6b-9598-4d42-9863-1b7f24cd90ba None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:34.157138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c762b6b-9598-4d42-9863-1b7f24cd90ba None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:34.157467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c762b6b-9598-4d42-9863-1b7f24cd90ba None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:34.157569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50fdc0-4e20-4073-9d4f-400e7c08fa16 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:34.159063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 159/310] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:33 2022] GET /identity/v3/users/7535b31484054f789e4df1d330e3491b => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:34.159194 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 151/310] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:04:34.167460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c762b6b-9598-4d42-9863-1b7f24cd90ba None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:34.168172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c02a0-9342-44fe-8f93-ae29ff194ba4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:34.168604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c02a0-9342-44fe-8f93-ae29ff194ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:34.169110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c02a0-9342-44fe-8f93-ae29ff194ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:34.200959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c23c02a0-9342-44fe-8f93-ae29ff194ba4 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:34.203606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c23c02a0-9342-44fe-8f93-ae29ff194ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:34.304632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 160/311] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:34.337049 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c762b6b-9598-4d42-9863-1b7f24cd90ba None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:34.375994 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54eacb-cb6b-4708-8ff3-1ea8111dd3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:34.376781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54eacb-cb6b-4708-8ff3-1ea8111dd3a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:34.377063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54eacb-cb6b-4708-8ff3-1ea8111dd3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:34.377391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54eacb-cb6b-4708-8ff3-1ea8111dd3a3 None admin] PATH_INFO: `/v3/system/users/7535b31484054f789e4df1d330e3491b/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:34.379038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54eacb-cb6b-4708-8ff3-1ea8111dd3a3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7535b31484054f789e4df1d330e3491b, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:34.381071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54eacb-cb6b-4708-8ff3-1ea8111dd3a3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:34.385663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:34.505233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 152/312] 10.222.0.83 () {58 vars in 1334 bytes} [Wed Aug 24 11:04:34 2022] PUT /identity/v3/system/users/7535b31484054f789e4df1d330e3491b/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:34.596792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c02a0-9342-44fe-8f93-ae29ff194ba4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:34.644325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:34.717108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c02a0-9342-44fe-8f93-ae29ff194ba4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:34.756492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0d3ee-3975-4e13-8ca2-c697c03d3052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:34.757550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0d3ee-3975-4e13-8ca2-c697c03d3052 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:34.758057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0d3ee-3975-4e13-8ca2-c697c03d3052 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:34.758523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0d3ee-3975-4e13-8ca2-c697c03d3052 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:34.771660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0d3ee-3975-4e13-8ca2-c697c03d3052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-00d0d3ee-3975-4e13-8ca2-c697c03d3052 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:34.777014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:34.777947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:34.778745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:34.781366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 161/313] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:34.787713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54eacb-cb6b-4708-8ff3-1ea8111dd3a3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:34.904472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54eacb-cb6b-4708-8ff3-1ea8111dd3a3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:34.973257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba79d0b-30e7-464d-a823-e13103439049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:34.974357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba79d0b-30e7-464d-a823-e13103439049 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:34.974857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba79d0b-30e7-464d-a823-e13103439049 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:34.975368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba79d0b-30e7-464d-a823-e13103439049 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:34.976647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba79d0b-30e7-464d-a823-e13103439049 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:34.977203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba79d0b-30e7-464d-a823-e13103439049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:34.980147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba79d0b-30e7-464d-a823-e13103439049 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:34.992614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 153/314] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:34.998469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0d3ee-3975-4e13-8ca2-c697c03d3052 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:35.104243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:35.108667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0d3ee-3975-4e13-8ca2-c697c03d3052 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:35.146447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51da26b-d6a3-472b-8d74-638cc89168fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:35.147480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51da26b-d6a3-472b-8d74-638cc89168fe None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:35.148013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51da26b-d6a3-472b-8d74-638cc89168fe None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:35.148486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51da26b-d6a3-472b-8d74-638cc89168fe None admin] PATH_INFO: `/v3/users/d2371d02ad9f40d8ad16ff498620df9d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:35.149960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51da26b-d6a3-472b-8d74-638cc89168fe None admin] RBAC: Authorizing `identity:get_user(user_id=d2371d02ad9f40d8ad16ff498620df9d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:35.151769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51da26b-d6a3-472b-8d74-638cc89168fe None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:35.154154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 162/315] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:34 2022] GET /identity/v3/users/d2371d02ad9f40d8ad16ff498620df9d => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:35.160818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba79d0b-30e7-464d-a823-e13103439049 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:35.277838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:35.331289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba79d0b-30e7-464d-a823-e13103439049 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:35.375328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f6894-45ca-4f5c-ab9d-f7414cd7e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:35.375983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f6894-45ca-4f5c-ab9d-f7414cd7e353 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:35.376219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f6894-45ca-4f5c-ab9d-f7414cd7e353 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:35.376496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f6894-45ca-4f5c-ab9d-f7414cd7e353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:35.456806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f6894-45ca-4f5c-ab9d-f7414cd7e353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:35.458275 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f6894-45ca-4f5c-ab9d-f7414cd7e353 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:35.529272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:35.574159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 154/316] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:35.579808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51da26b-d6a3-472b-8d74-638cc89168fe None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:35.696108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51da26b-d6a3-472b-8d74-638cc89168fe None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:35.737933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16683769-6705-4a22-956b-f2508ec9c8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:35.739441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16683769-6705-4a22-956b-f2508ec9c8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:35.739927 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16683769-6705-4a22-956b-f2508ec9c8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:35.740406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16683769-6705-4a22-956b-f2508ec9c8c8 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:35.741771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16683769-6705-4a22-956b-f2508ec9c8c8 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:35.743450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16683769-6705-4a22-956b-f2508ec9c8c8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:35.745696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 163/317] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:35 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:35.752449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f6894-45ca-4f5c-ab9d-f7414cd7e353 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:35.842723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26304fbf-668b-4531-b887-8ce50c5c5997 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:35.843051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26304fbf-668b-4531-b887-8ce50c5c5997 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:35.843272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26304fbf-668b-4531-b887-8ce50c5c5997 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:35.845980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 164/318] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:04:35.850467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543424-a322-4cb6-966d-a73a102db5e9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:35.850742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543424-a322-4cb6-966d-a73a102db5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:35.851015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543424-a322-4cb6-966d-a73a102db5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:35.871935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-21543424-a322-4cb6-966d-a73a102db5e9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:35.873139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-21543424-a322-4cb6-966d-a73a102db5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:35.873485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f6894-45ca-4f5c-ab9d-f7414cd7e353 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:35.906338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:35.911164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50846dde-9477-481c-87df-fcb1324d1325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:35.911783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50846dde-9477-481c-87df-fcb1324d1325 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:35.912069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50846dde-9477-481c-87df-fcb1324d1325 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:35.912353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50846dde-9477-481c-87df-fcb1324d1325 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:35.913100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50846dde-9477-481c-87df-fcb1324d1325 None admin] RBAC: Adding query filter params (role.id=0f2f4356fa294a7db98c5636fe2d390d, scope.project.id=858cb936469e4382b567530855fa7507, user.id=d2371d02ad9f40d8ad16ff498620df9d) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:35.913411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50846dde-9477-481c-87df-fcb1324d1325 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:35.914910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50846dde-9477-481c-87df-fcb1324d1325 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:35.922157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 155/319] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:04:35 2022] GET /identity/v3/role_assignments?user.id=d2371d02ad9f40d8ad16ff498620df9d&scope.project.id=858cb936469e4382b567530855fa7507&role.id=0f2f4356fa294a7db98c5636fe2d390d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:35.958099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 165/320] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:36.216587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e321-40a5-4371-aca0-d60d93c03663 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:36.216964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e321-40a5-4371-aca0-d60d93c03663 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:36.217141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e321-40a5-4371-aca0-d60d93c03663 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:36.218671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 156/321] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:36.223858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9f4cd-e652-42da-85ef-ba4381c59a4c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:36.224145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9f4cd-e652-42da-85ef-ba4381c59a4c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:36.224424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9f4cd-e652-42da-85ef-ba4381c59a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:36.227516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e321-40a5-4371-aca0-d60d93c03663 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:36.243033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ddd9f4cd-e652-42da-85ef-ba4381c59a4c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:36.244234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ddd9f4cd-e652-42da-85ef-ba4381c59a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:36.256925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:36.329284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 166/322] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:36.354710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e321-40a5-4371-aca0-d60d93c03663 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:36.390636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4c529-5546-4d3c-81e6-3c102b0886a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:36.391432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4c529-5546-4d3c-81e6-3c102b0886a7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:36.391663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4c529-5546-4d3c-81e6-3c102b0886a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:36.391935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4c529-5546-4d3c-81e6-3c102b0886a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:36.393711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4c529-5546-4d3c-81e6-3c102b0886a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:36.395190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4c529-5546-4d3c-81e6-3c102b0886a7 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:36.396944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 157/323] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:36.402444 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9f4cd-e652-42da-85ef-ba4381c59a4c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:36.502893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:36.510055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9f4cd-e652-42da-85ef-ba4381c59a4c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:36.547113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56125f9-25b5-4d50-89fa-2db5c650dd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:36.547763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56125f9-25b5-4d50-89fa-2db5c650dd4d None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:36.548007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56125f9-25b5-4d50-89fa-2db5c650dd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:36.548279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56125f9-25b5-4d50-89fa-2db5c650dd4d None admin] PATH_INFO: `/v3/groups` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:36.549465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56125f9-25b5-4d50-89fa-2db5c650dd4d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:36.550941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56125f9-25b5-4d50-89fa-2db5c650dd4d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:36.598787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4c529-5546-4d3c-81e6-3c102b0886a7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:36.674008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 167/324] 10.222.0.83 () {60 vars in 1134 bytes} [Wed Aug 24 11:04:36 2022] POST /identity/v3/groups => generated 226 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:36.675317 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaed809-a350-4298-a3a6-6900e3002070 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:36.675606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaed809-a350-4298-a3a6-6900e3002070 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:36.675888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaed809-a350-4298-a3a6-6900e3002070 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:36.677370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 168/325] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:36.683649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb6ef7-5228-45dd-ab57-7f33e7b7212a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:36.683992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb6ef7-5228-45dd-ab57-7f33e7b7212a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:36.684213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb6ef7-5228-45dd-ab57-7f33e7b7212a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:36.704300 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-71bb6ef7-5228-45dd-ab57-7f33e7b7212a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:36.705576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-71bb6ef7-5228-45dd-ab57-7f33e7b7212a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:36.705712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:36.714079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4c529-5546-4d3c-81e6-3c102b0886a7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:36.757539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80442ed-70de-475c-8b45-41feecc12772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:36.758130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80442ed-70de-475c-8b45-41feecc12772 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:36.758411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80442ed-70de-475c-8b45-41feecc12772 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:36.758708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80442ed-70de-475c-8b45-41feecc12772 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:36.774802 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80442ed-70de-475c-8b45-41feecc12772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e80442ed-70de-475c-8b45-41feecc12772 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:36.780525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:36.781615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:36.782614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 158/326] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:04:36.789561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80442ed-70de-475c-8b45-41feecc12772 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:36.815014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 169/327] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:36.925819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80442ed-70de-475c-8b45-41feecc12772 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:36.971487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51425e10-336d-47a6-84f5-2a237c48b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:36.972111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51425e10-336d-47a6-84f5-2a237c48b471 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:36.972397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51425e10-336d-47a6-84f5-2a237c48b471 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:36.972670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51425e10-336d-47a6-84f5-2a237c48b471 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:36.973391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51425e10-336d-47a6-84f5-2a237c48b471 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:36.973697 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51425e10-336d-47a6-84f5-2a237c48b471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:36.975121 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51425e10-336d-47a6-84f5-2a237c48b471 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:36.984857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 159/328] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:36.991341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb6ef7-5228-45dd-ab57-7f33e7b7212a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:37.091018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51425e10-336d-47a6-84f5-2a237c48b471 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:37.097264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:37.121520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:37.122122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb6ef7-5228-45dd-ab57-7f33e7b7212a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:37.159978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321a2e2-4e05-4292-a87e-b4ae1ed7e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:37.160801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321a2e2-4e05-4292-a87e-b4ae1ed7e2ee None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:37.160910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321a2e2-4e05-4292-a87e-b4ae1ed7e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:37.161171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321a2e2-4e05-4292-a87e-b4ae1ed7e2ee None admin] PATH_INFO: `/v3/users/c7cc7f5026e244629e66bad847def866` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:37.162406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321a2e2-4e05-4292-a87e-b4ae1ed7e2ee None admin] RBAC: Authorizing `identity:get_user(user_id=c7cc7f5026e244629e66bad847def866)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:37.163885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321a2e2-4e05-4292-a87e-b4ae1ed7e2ee None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:37.165955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 170/329] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:36 2022] GET /identity/v3/users/c7cc7f5026e244629e66bad847def866 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:37.171711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321a2e2-4e05-4292-a87e-b4ae1ed7e2ee None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:37.284051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51425e10-336d-47a6-84f5-2a237c48b471 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:37.311770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321a2e2-4e05-4292-a87e-b4ae1ed7e2ee None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:37.350023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e852cd-560e-4336-8257-d12c432e23aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:37.350600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e852cd-560e-4336-8257-d12c432e23aa None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:37.350880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e852cd-560e-4336-8257-d12c432e23aa None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:37.351157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e852cd-560e-4336-8257-d12c432e23aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:37.359932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ed5ad-b3e4-4de3-8b64-cefdc3f42c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:37.360828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ed5ad-b3e4-4de3-8b64-cefdc3f42c89 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:37.361899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ed5ad-b3e4-4de3-8b64-cefdc3f42c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:37.362372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ed5ad-b3e4-4de3-8b64-cefdc3f42c89 None admin] PATH_INFO: `/v3/system/users/c7cc7f5026e244629e66bad847def866/roles/0f2f4356fa294a7db98c5636fe2d390d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:37.364280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ed5ad-b3e4-4de3-8b64-cefdc3f42c89 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c7cc7f5026e244629e66bad847def866, role_id=0f2f4356fa294a7db98c5636fe2d390d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:37.365983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ed5ad-b3e4-4de3-8b64-cefdc3f42c89 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:37.367410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e852cd-560e-4336-8257-d12c432e23aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-52e852cd-560e-4336-8257-d12c432e23aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:37.373214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:37.374181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:37.374947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 160/330] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:37.380966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e852cd-560e-4336-8257-d12c432e23aa None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:37.447816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 171/331] 10.222.0.83 () {58 vars in 1334 bytes} [Wed Aug 24 11:04:37 2022] PUT /identity/v3/system/users/c7cc7f5026e244629e66bad847def866/roles/0f2f4356fa294a7db98c5636fe2d390d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:04:37.490861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:37.516863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e852cd-560e-4336-8257-d12c432e23aa None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:37.563892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8c6a4-015b-4e65-aabe-8fa3e5f29f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:37.565078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8c6a4-015b-4e65-aabe-8fa3e5f29f67 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:37.565360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8c6a4-015b-4e65-aabe-8fa3e5f29f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:37.565630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8c6a4-015b-4e65-aabe-8fa3e5f29f67 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:37.566380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8c6a4-015b-4e65-aabe-8fa3e5f29f67 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:37.566687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8c6a4-015b-4e65-aabe-8fa3e5f29f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:37.568095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8c6a4-015b-4e65-aabe-8fa3e5f29f67 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:37.577805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 161/332] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:37.583667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ed5ad-b3e4-4de3-8b64-cefdc3f42c89 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:37.613786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:37.717624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ed5ad-b3e4-4de3-8b64-cefdc3f42c89 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:37.756966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bf5e7-8439-4e9b-bed7-42c2f23092b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:37.757617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bf5e7-8439-4e9b-bed7-42c2f23092b8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:37.757934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bf5e7-8439-4e9b-bed7-42c2f23092b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:37.758185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bf5e7-8439-4e9b-bed7-42c2f23092b8 None admin] PATH_INFO: `/v3/users/7535b31484054f789e4df1d330e3491b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:37.759337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8bf5e7-8439-4e9b-bed7-42c2f23092b8 None admin] RBAC: Authorizing `identity:get_user(user_id=7535b31484054f789e4df1d330e3491b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:37.760806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8bf5e7-8439-4e9b-bed7-42c2f23092b8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:37.762806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 172/333] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:37 2022] GET /identity/v3/users/7535b31484054f789e4df1d330e3491b => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:37.769000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8c6a4-015b-4e65-aabe-8fa3e5f29f67 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:37.918334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8c6a4-015b-4e65-aabe-8fa3e5f29f67 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:38.006098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9d8e9-71af-4021-9666-10a98e39c0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:38.007138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9d8e9-71af-4021-9666-10a98e39c0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:38.007645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9d8e9-71af-4021-9666-10a98e39c0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:38.008109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9d8e9-71af-4021-9666-10a98e39c0b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:38.009534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9d8e9-71af-4021-9666-10a98e39c0b6 None admin] RBAC: Adding query filter params (role.id=799b2f3554434281bf10b2602fbbc09a, user.id=7535b31484054f789e4df1d330e3491b, scope.system=all) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:38.010136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9d8e9-71af-4021-9666-10a98e39c0b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:38.012875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9d8e9-71af-4021-9666-10a98e39c0b6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:38.020432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:38.039246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 162/334] 10.222.0.83 () {56 vars in 1308 bytes} [Wed Aug 24 11:04:37 2022] GET /identity/v3/role_assignments?user.id=7535b31484054f789e4df1d330e3491b&scope.system=all&role.id=799b2f3554434281bf10b2602fbbc09a => generated 510 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:39.044620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35da8d-ea91-4a8e-b24f-0e02c6e85d38 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:39.045166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35da8d-ea91-4a8e-b24f-0e02c6e85d38 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:39.045166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35da8d-ea91-4a8e-b24f-0e02c6e85d38 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:39.046947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 173/335] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:39.054229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932968f5-f82d-47b7-8e23-ff731c4b2abb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:39.054720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932968f5-f82d-47b7-8e23-ff731c4b2abb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:39.056979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932968f5-f82d-47b7-8e23-ff731c4b2abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:39.076552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-932968f5-f82d-47b7-8e23-ff731c4b2abb None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:39.077861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-932968f5-f82d-47b7-8e23-ff731c4b2abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:39.164708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 163/336] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:39.747526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35da8d-ea91-4a8e-b24f-0e02c6e85d38 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:39.861120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:39.881741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35da8d-ea91-4a8e-b24f-0e02c6e85d38 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:39.920532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a07187-6bfb-4d20-b864-1efffc8e5894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:39.921173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a07187-6bfb-4d20-b864-1efffc8e5894 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:39.921459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a07187-6bfb-4d20-b864-1efffc8e5894 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:39.921739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a07187-6bfb-4d20-b864-1efffc8e5894 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:39.936424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c548a-f622-43c3-bb98-c58ade93137e None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:39.936756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c548a-f622-43c3-bb98-c58ade93137e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:39.936960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c548a-f622-43c3-bb98-c58ade93137e None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:39.938403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 164/337] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:04:39.941842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a07187-6bfb-4d20-b864-1efffc8e5894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:39.944710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e9ff4-8ac7-4f84-aa29-4cc136883da0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:39.944992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e9ff4-8ac7-4f84-aa29-4cc136883da0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:39.945263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e9ff4-8ac7-4f84-aa29-4cc136883da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f6a07187-6bfb-4d20-b864-1efffc8e5894 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:39.946841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:39.947761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:39.948590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:39.952839 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 174/338] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:39.954154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a07187-6bfb-4d20-b864-1efffc8e5894 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:39.967860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d91e9ff4-8ac7-4f84-aa29-4cc136883da0 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:39.969075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d91e9ff4-8ac7-4f84-aa29-4cc136883da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:40.071264 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 165/339] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:40.089154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a07187-6bfb-4d20-b864-1efffc8e5894 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:40.125953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d72450-1876-4c84-84c0-9145f1a4fff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:40.127124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d72450-1876-4c84-84c0-9145f1a4fff9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:40.127403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d72450-1876-4c84-84c0-9145f1a4fff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:40.127672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d72450-1876-4c84-84c0-9145f1a4fff9 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:40.128362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d72450-1876-4c84-84c0-9145f1a4fff9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:40.128666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d72450-1876-4c84-84c0-9145f1a4fff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:40.130130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d72450-1876-4c84-84c0-9145f1a4fff9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:40.141392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 175/340] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:40.146164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e9ff4-8ac7-4f84-aa29-4cc136883da0 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:40.238339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:40.258640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e9ff4-8ac7-4f84-aa29-4cc136883da0 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:40.296500 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc353-2043-459d-97ba-739f93627534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:40.297127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc353-2043-459d-97ba-739f93627534 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:40.297393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc353-2043-459d-97ba-739f93627534 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:40.297849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc353-2043-459d-97ba-739f93627534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:40.345520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076085e5-c902-42bb-9c72-2aee17bbdf83 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:40.345834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076085e5-c902-42bb-9c72-2aee17bbdf83 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:40.346088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076085e5-c902-42bb-9c72-2aee17bbdf83 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:40.347831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 176/341] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:40.353571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ced27-ae58-44cb-b2e9-17259495c908 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:40.353896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ced27-ae58-44cb-b2e9-17259495c908 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:40.354146 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ced27-ae58-44cb-b2e9-17259495c908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:40.374501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:40.390395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cc353-2043-459d-97ba-739f93627534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:40.391885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cc353-2043-459d-97ba-739f93627534 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:40.398102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c94ced27-ae58-44cb-b2e9-17259495c908 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:40.399358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c94ced27-ae58-44cb-b2e9-17259495c908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:40.460383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:40.515523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 166/342] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:40.516921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2f813-bc30-47ff-b8d2-b9e085aa683b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:40.517924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2f813-bc30-47ff-b8d2-b9e085aa683b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:40.518337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2f813-bc30-47ff-b8d2-b9e085aa683b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:40.521119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 167/343] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:40.525200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2f813-bc30-47ff-b8d2-b9e085aa683b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:40.584352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 177/344] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:40.586201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4679e-4b5e-4f76-9098-89167f8e2016 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:40.586642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4679e-4b5e-4f76-9098-89167f8e2016 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:40.587047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4679e-4b5e-4f76-9098-89167f8e2016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:40.612843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-48e4679e-4b5e-4f76-9098-89167f8e2016 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:40.614083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-48e4679e-4b5e-4f76-9098-89167f8e2016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:40.708437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 178/345] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:40.709324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4679e-4b5e-4f76-9098-89167f8e2016 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:40.767025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2f813-bc30-47ff-b8d2-b9e085aa683b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:40.832403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:40.851738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe56a2-f16a-4e89-b4ed-7519da68e3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:40.852529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe56a2-f16a-4e89-b4ed-7519da68e3ca None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:40.858358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe56a2-f16a-4e89-b4ed-7519da68e3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:40.859189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe56a2-f16a-4e89-b4ed-7519da68e3ca None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:40.860945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe56a2-f16a-4e89-b4ed-7519da68e3ca None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:40.863140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe56a2-f16a-4e89-b4ed-7519da68e3ca None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:40.865338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 168/346] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:40 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:40.877350 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe56a2-f16a-4e89-b4ed-7519da68e3ca None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:40.918711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:40.936701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4679e-4b5e-4f76-9098-89167f8e2016 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:41.015331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094239d-e4b8-463c-a4b2-0f149a5b4aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:41.016323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094239d-e4b8-463c-a4b2-0f149a5b4aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.016776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094239d-e4b8-463c-a4b2-0f149a5b4aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.017748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094239d-e4b8-463c-a4b2-0f149a5b4aa8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.037915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094239d-e4b8-463c-a4b2-0f149a5b4aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:41.047321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe56a2-f16a-4e89-b4ed-7519da68e3ca None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e094239d-e4b8-463c-a4b2-0f149a5b4aa8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:41.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:41.050424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 179/347] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:41.060369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094239d-e4b8-463c-a4b2-0f149a5b4aa8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:41.114115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac522-1a8a-4484-b15e-600b2feeb8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:41.115342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac522-1a8a-4484-b15e-600b2feeb8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.115752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac522-1a8a-4484-b15e-600b2feeb8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.116161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac522-1a8a-4484-b15e-600b2feeb8f8 None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.125903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ac522-1a8a-4484-b15e-600b2feeb8f8 None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:41.127720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ac522-1a8a-4484-b15e-600b2feeb8f8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:41.130562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 169/348] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:40 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:41.150256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac522-1a8a-4484-b15e-600b2feeb8f8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:41.188513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094239d-e4b8-463c-a4b2-0f149a5b4aa8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:41.222217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:41.237660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adc150-9dde-496c-8ab1-31a67cb33b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:41.238640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adc150-9dde-496c-8ab1-31a67cb33b63 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.240267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adc150-9dde-496c-8ab1-31a67cb33b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.240734 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adc150-9dde-496c-8ab1-31a67cb33b63 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.245329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64adc150-9dde-496c-8ab1-31a67cb33b63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:41.245969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64adc150-9dde-496c-8ab1-31a67cb33b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:41.247808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64adc150-9dde-496c-8ab1-31a67cb33b63 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:41.268935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 180/349] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:41.271116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adc150-9dde-496c-8ab1-31a67cb33b63 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:41.340252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:41.364496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac522-1a8a-4484-b15e-600b2feeb8f8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:41.429157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adc150-9dde-496c-8ab1-31a67cb33b63 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:41.433740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a02e7-2d2c-4553-aa99-972ccf399d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:41.435522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a02e7-2d2c-4553-aa99-972ccf399d23 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.436372 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a02e7-2d2c-4553-aa99-972ccf399d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.437115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a02e7-2d2c-4553-aa99-972ccf399d23 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.456220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a02e7-2d2c-4553-aa99-972ccf399d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-3f9a02e7-2d2c-4553-aa99-972ccf399d23 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:41.464282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:41.465652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:41.466529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:41.469334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 170/350] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:04:41.471158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a02e7-2d2c-4553-aa99-972ccf399d23 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:41.487596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123d797-6347-4306-9f4c-b4a72cf9943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:41.488238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123d797-6347-4306-9f4c-b4a72cf9943c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.488563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123d797-6347-4306-9f4c-b4a72cf9943c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.488843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123d797-6347-4306-9f4c-b4a72cf9943c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.489592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123d797-6347-4306-9f4c-b4a72cf9943c None admin] RBAC: Adding query filter params (group.id=31d7b063f5f44104891f1babd3641dbc, role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:41.489924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123d797-6347-4306-9f4c-b4a72cf9943c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:41.492198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123d797-6347-4306-9f4c-b4a72cf9943c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:41.500409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 181/351] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:41 2022] GET /identity/v3/role_assignments?group.id=31d7b063f5f44104891f1babd3641dbc&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=799b2f3554434281bf10b2602fbbc09a => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:41.504169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123d797-6347-4306-9f4c-b4a72cf9943c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:41.598514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:41.605467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a02e7-2d2c-4553-aa99-972ccf399d23 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:41.654429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b00a6-0251-4683-9751-a9de8e31afac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:41.655337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b00a6-0251-4683-9751-a9de8e31afac None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.655783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b00a6-0251-4683-9751-a9de8e31afac None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.656195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b00a6-0251-4683-9751-a9de8e31afac None admin] PATH_INFO: `/v3/users/c7cc7f5026e244629e66bad847def866` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.657591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b00a6-0251-4683-9751-a9de8e31afac None admin] RBAC: Authorizing `identity:get_user(user_id=c7cc7f5026e244629e66bad847def866)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:41.659524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b00a6-0251-4683-9751-a9de8e31afac None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:41.662139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 171/352] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:04:41 2022] GET /identity/v3/users/c7cc7f5026e244629e66bad847def866 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:41.663319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b00a6-0251-4683-9751-a9de8e31afac None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:41.684996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123d797-6347-4306-9f4c-b4a72cf9943c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:41.701150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:41.765592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd22cf-4524-40d0-9dd0-9c6805764396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:41.767377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd22cf-4524-40d0-9dd0-9c6805764396 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.767868 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd22cf-4524-40d0-9dd0-9c6805764396 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.768303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd22cf-4524-40d0-9dd0-9c6805764396 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.791222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd22cf-4524-40d0-9dd0-9c6805764396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-93cd22cf-4524-40d0-9dd0-9c6805764396 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:41.806701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:41.809021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:41.810847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:41.816519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 182/353] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:41.819388 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd22cf-4524-40d0-9dd0-9c6805764396 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:41.906077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b00a6-0251-4683-9751-a9de8e31afac None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:41.951300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f2f70-fffd-4b77-a8f2-f533720f9775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:41.952070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f2f70-fffd-4b77-a8f2-f533720f9775 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.952397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f2f70-fffd-4b77-a8f2-f533720f9775 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.952791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f2f70-fffd-4b77-a8f2-f533720f9775 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.953564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f2f70-fffd-4b77-a8f2-f533720f9775 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:41.954074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f2f70-fffd-4b77-a8f2-f533720f9775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:41.956574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f2f70-fffd-4b77-a8f2-f533720f9775 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:41.967354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 172/354] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:41.970720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8dfab-46ec-450c-9dbd-7aa2b9cd17d1 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:41.971055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8dfab-46ec-450c-9dbd-7aa2b9cd17d1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:41.971569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8dfab-46ec-450c-9dbd-7aa2b9cd17d1 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:41.974499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 173/355] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:41.976102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8dfab-46ec-450c-9dbd-7aa2b9cd17d1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:41.984284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd22cf-4524-40d0-9dd0-9c6805764396 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:42.000305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:42.025032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c960fa0-e3a0-4a18-b185-92b0be7aa317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:42.025752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c960fa0-e3a0-4a18-b185-92b0be7aa317 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.026184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c960fa0-e3a0-4a18-b185-92b0be7aa317 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.026528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c960fa0-e3a0-4a18-b185-92b0be7aa317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.027469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c960fa0-e3a0-4a18-b185-92b0be7aa317 None admin] RBAC: Adding query filter params (role.id=0f2f4356fa294a7db98c5636fe2d390d, user.id=c7cc7f5026e244629e66bad847def866, scope.system=all) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:42.027910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c960fa0-e3a0-4a18-b185-92b0be7aa317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:42.029526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c960fa0-e3a0-4a18-b185-92b0be7aa317 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:42.047246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 183/356] 10.222.0.83 () {56 vars in 1308 bytes} [Wed Aug 24 11:04:41 2022] GET /identity/v3/role_assignments?user.id=c7cc7f5026e244629e66bad847def866&scope.system=all&role.id=0f2f4356fa294a7db98c5636fe2d390d => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:42.048345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c960fa0-e3a0-4a18-b185-92b0be7aa317 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:42.102648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:42.147344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8dfab-46ec-450c-9dbd-7aa2b9cd17d1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:42.193383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac8150-97df-4041-8420-eb25dec3cc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:42.194587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac8150-97df-4041-8420-eb25dec3cc67 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.195159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac8150-97df-4041-8420-eb25dec3cc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.195651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac8150-97df-4041-8420-eb25dec3cc67 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.196589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac8150-97df-4041-8420-eb25dec3cc67 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:42.197114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac8150-97df-4041-8420-eb25dec3cc67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:42.199010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac8150-97df-4041-8420-eb25dec3cc67 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:42.201346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c960fa0-e3a0-4a18-b185-92b0be7aa317 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:42.211239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 174/357] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:42.213284 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e062847-9fc4-46dc-9f8e-a1a37ba4c824 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.215018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e062847-9fc4-46dc-9f8e-a1a37ba4c824 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.215638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e062847-9fc4-46dc-9f8e-a1a37ba4c824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.242918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7e062847-9fc4-46dc-9f8e-a1a37ba4c824 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:42.244493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7e062847-9fc4-46dc-9f8e-a1a37ba4c824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:42.249022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8400a5-ddf3-4d0c-aee3-6ef55da48b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:42.249022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8400a5-ddf3-4d0c-aee3-6ef55da48b9c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.249022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8400a5-ddf3-4d0c-aee3-6ef55da48b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.249022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8400a5-ddf3-4d0c-aee3-6ef55da48b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.300550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:42.358175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8400a5-ddf3-4d0c-aee3-6ef55da48b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:42.359017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 175/358] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:42.359693 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8400a5-ddf3-4d0c-aee3-6ef55da48b9c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:42.360361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cafd64-82ad-4321-9e21-2653100fc818 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.360787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cafd64-82ad-4321-9e21-2653100fc818 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.361707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cafd64-82ad-4321-9e21-2653100fc818 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.362690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 176/359] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:42.363426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cafd64-82ad-4321-9e21-2653100fc818 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:42.472380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:42.510819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 184/360] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:42.511810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cafd64-82ad-4321-9e21-2653100fc818 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:42.512229 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af068fdc-c223-4fcd-9ef6-d6b19e525535 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.512515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af068fdc-c223-4fcd-9ef6-d6b19e525535 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.512790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af068fdc-c223-4fcd-9ef6-d6b19e525535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.532214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-af068fdc-c223-4fcd-9ef6-d6b19e525535 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:42.534140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-af068fdc-c223-4fcd-9ef6-d6b19e525535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:42.570465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea165a4-4bcb-4bf7-93bf-321ba6e1f2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:42.571088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea165a4-4bcb-4bf7-93bf-321ba6e1f2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.571370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea165a4-4bcb-4bf7-93bf-321ba6e1f2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.571645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea165a4-4bcb-4bf7-93bf-321ba6e1f2a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.644508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:42.652410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 185/361] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:42.653812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af068fdc-c223-4fcd-9ef6-d6b19e525535 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:42.670955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea165a4-4bcb-4bf7-93bf-321ba6e1f2a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:42.674142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea165a4-4bcb-4bf7-93bf-321ba6e1f2a6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:42.761036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:42.783512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af068fdc-c223-4fcd-9ef6-d6b19e525535 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:42.803203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 177/362] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:42.805149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea165a4-4bcb-4bf7-93bf-321ba6e1f2a6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:42.837352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05c54d-9ebd-4f54-8361-ebc28b7785b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:42.838227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05c54d-9ebd-4f54-8361-ebc28b7785b3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:42.838629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05c54d-9ebd-4f54-8361-ebc28b7785b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:42.838910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05c54d-9ebd-4f54-8361-ebc28b7785b3 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:42.840088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05c54d-9ebd-4f54-8361-ebc28b7785b3 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:42.842112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05c54d-9ebd-4f54-8361-ebc28b7785b3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:42.844888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 186/363] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:42 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:42.846256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05c54d-9ebd-4f54-8361-ebc28b7785b3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:42.939859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:42.958644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea165a4-4bcb-4bf7-93bf-321ba6e1f2a6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:42.990999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05c54d-9ebd-4f54-8361-ebc28b7785b3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:43.007224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f2d22-438f-4445-80d9-71a3fa795914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.008128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f2d22-438f-4445-80d9-71a3fa795914 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.008437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f2d22-438f-4445-80d9-71a3fa795914 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.008692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f2d22-438f-4445-80d9-71a3fa795914 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.022474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f2d22-438f-4445-80d9-71a3fa795914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-cf2f2d22-438f-4445-80d9-71a3fa795914 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:43.027360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:43.028286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 178/364] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:43.030849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f2d22-438f-4445-80d9-71a3fa795914 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:43.040858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e95583-d41e-4a4d-8928-caba9c4659dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.041499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e95583-d41e-4a4d-8928-caba9c4659dc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.042096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e95583-d41e-4a4d-8928-caba9c4659dc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.042228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e95583-d41e-4a4d-8928-caba9c4659dc None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.043434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e95583-d41e-4a4d-8928-caba9c4659dc None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:43.044843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e95583-d41e-4a4d-8928-caba9c4659dc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:43.049061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 187/365] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:42 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:43.049061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e95583-d41e-4a4d-8928-caba9c4659dc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:43.081192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:43.175014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f2d22-438f-4445-80d9-71a3fa795914 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:43.195911 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e95583-d41e-4a4d-8928-caba9c4659dc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:43.225205 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d0ca4-e566-4412-8403-7233eeda06e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.226471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d0ca4-e566-4412-8403-7233eeda06e2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.226928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d0ca4-e566-4412-8403-7233eeda06e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.227491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d0ca4-e566-4412-8403-7233eeda06e2 None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.229109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d0ca4-e566-4412-8403-7233eeda06e2 None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:43.231237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d0ca4-e566-4412-8403-7233eeda06e2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:43.234128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 179/366] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:43.235160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d0ca4-e566-4412-8403-7233eeda06e2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:43.245801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a7a45-2570-4221-a9cb-255962d87e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.246517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a7a45-2570-4221-a9cb-255962d87e82 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.246802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a7a45-2570-4221-a9cb-255962d87e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.247080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a7a45-2570-4221-a9cb-255962d87e82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.256694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:43.262886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a7a45-2570-4221-a9cb-255962d87e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-641a7a45-2570-4221-a9cb-255962d87e82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:43.268610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:43.274331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 188/367] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:43.275258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a7a45-2570-4221-a9cb-255962d87e82 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:43.365076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:43.369533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d0ca4-e566-4412-8403-7233eeda06e2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:43.418042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590ac3e-6abf-4e9b-9bab-b8bfd3e36646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.418650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590ac3e-6abf-4e9b-9bab-b8bfd3e36646 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.418935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590ac3e-6abf-4e9b-9bab-b8bfd3e36646 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.419200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590ac3e-6abf-4e9b-9bab-b8bfd3e36646 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.419885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590ac3e-6abf-4e9b-9bab-b8bfd3e36646 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:43.420190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590ac3e-6abf-4e9b-9bab-b8bfd3e36646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:43.421564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590ac3e-6abf-4e9b-9bab-b8bfd3e36646 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:43.421802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a7a45-2570-4221-a9cb-255962d87e82 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:43.433708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 180/368] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:43.434656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590ac3e-6abf-4e9b-9bab-b8bfd3e36646 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:43.477510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f276a-4cbc-40d8-88bf-cbc2c7e95214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.478805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f276a-4cbc-40d8-88bf-cbc2c7e95214 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.478805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f276a-4cbc-40d8-88bf-cbc2c7e95214 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.478805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f276a-4cbc-40d8-88bf-cbc2c7e95214 None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.480664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f276a-4cbc-40d8-88bf-cbc2c7e95214 None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:43.482360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f276a-4cbc-40d8-88bf-cbc2c7e95214 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:43.483796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 189/369] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:43.484662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f276a-4cbc-40d8-88bf-cbc2c7e95214 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:43.564360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:43.576761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590ac3e-6abf-4e9b-9bab-b8bfd3e36646 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:43.617575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f276a-4cbc-40d8-88bf-cbc2c7e95214 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:43.635411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4bbb9-8e1c-4b3e-a946-3c789c304e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.636138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4bbb9-8e1c-4b3e-a946-3c789c304e98 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.636422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4bbb9-8e1c-4b3e-a946-3c789c304e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.636720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4bbb9-8e1c-4b3e-a946-3c789c304e98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.637455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4bbb9-8e1c-4b3e-a946-3c789c304e98 None admin] RBAC: Adding query filter params (group.id=31d7b063f5f44104891f1babd3641dbc, role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:43.637768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4bbb9-8e1c-4b3e-a946-3c789c304e98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:43.639294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4bbb9-8e1c-4b3e-a946-3c789c304e98 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:43.649506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 181/370] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3/role_assignments?group.id=31d7b063f5f44104891f1babd3641dbc&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:43.650320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4bbb9-8e1c-4b3e-a946-3c789c304e98 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:43.666016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bd97a-4706-42cc-a45b-ee5c174eec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.666650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bd97a-4706-42cc-a45b-ee5c174eec2e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.667048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bd97a-4706-42cc-a45b-ee5c174eec2e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.667326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bd97a-4706-42cc-a45b-ee5c174eec2e None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.668138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bd97a-4706-42cc-a45b-ee5c174eec2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:43.668440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bd97a-4706-42cc-a45b-ee5c174eec2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:43.669856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bd97a-4706-42cc-a45b-ee5c174eec2e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:43.679868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:43.681074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 190/371] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:43.682346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bd97a-4706-42cc-a45b-ee5c174eec2e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:43.789831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4bbb9-8e1c-4b3e-a946-3c789c304e98 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:43.843842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bd97a-4706-42cc-a45b-ee5c174eec2e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:43.844214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eee7b7-22de-48a8-9421-c1f3e6f63481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.845047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eee7b7-22de-48a8-9421-c1f3e6f63481 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.845524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eee7b7-22de-48a8-9421-c1f3e6f63481 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.845976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eee7b7-22de-48a8-9421-c1f3e6f63481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.888778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:43.900622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e36776-c3b9-4d75-b145-87e056323160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:43.905776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e36776-c3b9-4d75-b145-87e056323160 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:43.905776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e36776-c3b9-4d75-b145-87e056323160 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:43.905776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e36776-c3b9-4d75-b145-87e056323160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:43.905776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e36776-c3b9-4d75-b145-87e056323160 None admin] RBAC: Adding query filter params (group.id=31d7b063f5f44104891f1babd3641dbc, role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=858cb936469e4382b567530855fa7507) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:43.905776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e36776-c3b9-4d75-b145-87e056323160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:43.905776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e36776-c3b9-4d75-b145-87e056323160 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:43.911017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 191/372] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3/role_assignments?group.id=31d7b063f5f44104891f1babd3641dbc&scope.project.id=858cb936469e4382b567530855fa7507&role.id=799b2f3554434281bf10b2602fbbc09a => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:43.911995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e36776-c3b9-4d75-b145-87e056323160 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:44.025301 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eee7b7-22de-48a8-9421-c1f3e6f63481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:44.026839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eee7b7-22de-48a8-9421-c1f3e6f63481 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:44.053319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e36776-c3b9-4d75-b145-87e056323160 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:44.066339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:44.117498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52522ed0-597b-410b-a831-ef1feecc7150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:44.118395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522ed0-597b-410b-a831-ef1feecc7150 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:44.118826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522ed0-597b-410b-a831-ef1feecc7150 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:44.119223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522ed0-597b-410b-a831-ef1feecc7150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:44.186320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 182/373] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:44.187545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414ec4d-207e-4bf6-b022-b67daeba5538 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:44.187839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414ec4d-207e-4bf6-b022-b67daeba5538 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:44.188108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414ec4d-207e-4bf6-b022-b67daeba5538 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:44.189941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 183/374] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:44.194399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414ec4d-207e-4bf6-b022-b67daeba5538 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:44.208708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:44.225829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52522ed0-597b-410b-a831-ef1feecc7150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:44.228035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52522ed0-597b-410b-a831-ef1feecc7150 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:44.351093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414ec4d-207e-4bf6-b022-b67daeba5538 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:44.384135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:44.397525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 192/375] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:44.403267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b9104-c4c8-46e6-8b66-6f72ac4b8737 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:44.404566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b9104-c4c8-46e6-8b66-6f72ac4b8737 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:44.404566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b9104-c4c8-46e6-8b66-6f72ac4b8737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:44.405503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f4757-b690-4d9a-afbc-4a5c5e14c3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:44.406619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f4757-b690-4d9a-afbc-4a5c5e14c3db None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:44.406968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f4757-b690-4d9a-afbc-4a5c5e14c3db None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:44.407195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f4757-b690-4d9a-afbc-4a5c5e14c3db None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:44.408598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f4757-b690-4d9a-afbc-4a5c5e14c3db None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:44.410924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f4757-b690-4d9a-afbc-4a5c5e14c3db None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:44.412181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 184/376] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:44 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:44.413373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f4757-b690-4d9a-afbc-4a5c5e14c3db None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:44.426821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-331b9104-c4c8-46e6-8b66-6f72ac4b8737 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:44.428347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-331b9104-c4c8-46e6-8b66-6f72ac4b8737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:44.511714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 193/377] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:44.512923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b9104-c4c8-46e6-8b66-6f72ac4b8737 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:44.540358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:44.580771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f4757-b690-4d9a-afbc-4a5c5e14c3db None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:44.654717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7dd87-2992-456b-a86f-9df02b6332ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:44.655283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7dd87-2992-456b-a86f-9df02b6332ff None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:44.655565 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7dd87-2992-456b-a86f-9df02b6332ff None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:44.655844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7dd87-2992-456b-a86f-9df02b6332ff None admin] PATH_INFO: `/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:44.657071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7dd87-2992-456b-a86f-9df02b6332ff None admin] RBAC: Authorizing `identity:get_project(project_id=17f9dd8b207b49df9eb8e79eece61c9b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:44.659101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7dd87-2992-456b-a86f-9df02b6332ff None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:44.665203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 185/378] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:44 2022] GET /identity/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b => generated 344 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:44.668307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7dd87-2992-456b-a86f-9df02b6332ff None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:44.682321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b9104-c4c8-46e6-8b66-6f72ac4b8737 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:44.752826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060ad27-9d4a-414a-9afd-021c0f189bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:44.753482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060ad27-9d4a-414a-9afd-021c0f189bee None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:44.753762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060ad27-9d4a-414a-9afd-021c0f189bee None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:44.754068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060ad27-9d4a-414a-9afd-021c0f189bee None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:44.755483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060ad27-9d4a-414a-9afd-021c0f189bee None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:44.756963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060ad27-9d4a-414a-9afd-021c0f189bee None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:44.758609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 194/379] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:44 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:44.764636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060ad27-9d4a-414a-9afd-021c0f189bee None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:44.788585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:44.854666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7dd87-2992-456b-a86f-9df02b6332ff None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:44.906434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:44.907554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cac762-424b-43af-96bd-587eb8ad017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:44.908278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cac762-424b-43af-96bd-587eb8ad017c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:44.908482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cac762-424b-43af-96bd-587eb8ad017c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:44.908788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cac762-424b-43af-96bd-587eb8ad017c None admin] PATH_INFO: `/v3/groups/33851d52f05144f384cd05645ac63334` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:44.919093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cac762-424b-43af-96bd-587eb8ad017c None admin] RBAC: Authorizing `identity:get_group(group_id=33851d52f05144f384cd05645ac63334)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:44.925303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cac762-424b-43af-96bd-587eb8ad017c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:44.925303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 186/380] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:44 2022] GET /identity/v3/groups/33851d52f05144f384cd05645ac63334 => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:44.925303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cac762-424b-43af-96bd-587eb8ad017c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:44.928000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060ad27-9d4a-414a-9afd-021c0f189bee None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:44.982552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592548d-3cfb-465a-b414-6330acd1b4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:44.983968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592548d-3cfb-465a-b414-6330acd1b4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:44.984435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592548d-3cfb-465a-b414-6330acd1b4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:44.985304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592548d-3cfb-465a-b414-6330acd1b4a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:44.987219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592548d-3cfb-465a-b414-6330acd1b4a8 None admin] RBAC: Adding query filter params (group.id=31d7b063f5f44104891f1babd3641dbc, role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=858cb936469e4382b567530855fa7507) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:44.987545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592548d-3cfb-465a-b414-6330acd1b4a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:44.989036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592548d-3cfb-465a-b414-6330acd1b4a8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:44.995706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 195/381] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:44 2022] GET /identity/v3/role_assignments?group.id=31d7b063f5f44104891f1babd3641dbc&scope.project.id=858cb936469e4382b567530855fa7507&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:44.999470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592548d-3cfb-465a-b414-6330acd1b4a8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:45.077689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cac762-424b-43af-96bd-587eb8ad017c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:45.119958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:45.141836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e50cea-9d5c-4bd1-adcd-3c60299965f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:45.143747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e50cea-9d5c-4bd1-adcd-3c60299965f1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:45.144200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e50cea-9d5c-4bd1-adcd-3c60299965f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:45.144639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e50cea-9d5c-4bd1-adcd-3c60299965f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:45.145315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592548d-3cfb-465a-b414-6330acd1b4a8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:45.175499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e50cea-9d5c-4bd1-adcd-3c60299965f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-f1e50cea-9d5c-4bd1-adcd-3c60299965f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:45.183958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:45.184942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:45.185728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 187/382] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:45.192589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e50cea-9d5c-4bd1-adcd-3c60299965f1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:45.226289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1265d3f-81ef-40c5-a7e7-e8243ee0940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:45.226983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1265d3f-81ef-40c5-a7e7-e8243ee0940a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:45.227261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1265d3f-81ef-40c5-a7e7-e8243ee0940a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:45.227590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1265d3f-81ef-40c5-a7e7-e8243ee0940a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:45.228416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1265d3f-81ef-40c5-a7e7-e8243ee0940a None admin] RBAC: Adding query filter params (group.id=33851d52f05144f384cd05645ac63334, role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=17f9dd8b207b49df9eb8e79eece61c9b) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:45.228730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1265d3f-81ef-40c5-a7e7-e8243ee0940a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:45.232141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1265d3f-81ef-40c5-a7e7-e8243ee0940a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:45.244000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 196/383] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:44 2022] GET /identity/v3/role_assignments?group.id=33851d52f05144f384cd05645ac63334&scope.project.id=17f9dd8b207b49df9eb8e79eece61c9b&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:45.371673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e50cea-9d5c-4bd1-adcd-3c60299965f1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:45.431943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d8893-010b-4734-b209-45a0182020f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:45.432500 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d8893-010b-4734-b209-45a0182020f1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:45.432790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d8893-010b-4734-b209-45a0182020f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:45.433365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d8893-010b-4734-b209-45a0182020f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:45.434155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d8893-010b-4734-b209-45a0182020f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:45.434600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d8893-010b-4734-b209-45a0182020f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:45.436339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d8893-010b-4734-b209-45a0182020f1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:45.451161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 188/384] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:45.458048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1265d3f-81ef-40c5-a7e7-e8243ee0940a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:45.482139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:45.573802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1265d3f-81ef-40c5-a7e7-e8243ee0940a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:45.620142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730e6ab-3b12-4ab7-9683-6dcfc5badfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:45.620845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730e6ab-3b12-4ab7-9683-6dcfc5badfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:45.621126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730e6ab-3b12-4ab7-9683-6dcfc5badfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:45.621448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730e6ab-3b12-4ab7-9683-6dcfc5badfe2 None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:45.622626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730e6ab-3b12-4ab7-9683-6dcfc5badfe2 None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:45.624086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730e6ab-3b12-4ab7-9683-6dcfc5badfe2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:45.626316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 197/385] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:45 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:45.631821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d8893-010b-4734-b209-45a0182020f1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:45.697885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:45.770419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d8893-010b-4734-b209-45a0182020f1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:45.808477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0dd87-6e0b-4919-90eb-921dce848fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:45.809120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0dd87-6e0b-4919-90eb-921dce848fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:45.809399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0dd87-6e0b-4919-90eb-921dce848fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:45.809676 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0dd87-6e0b-4919-90eb-921dce848fa5 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:45.812453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0dd87-6e0b-4919-90eb-921dce848fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:45.814534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0dd87-6e0b-4919-90eb-921dce848fa5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:45.816534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 189/386] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:45 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:45.822242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730e6ab-3b12-4ab7-9683-6dcfc5badfe2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:45.914452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:45.932028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730e6ab-3b12-4ab7-9683-6dcfc5badfe2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:45.969668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a0df3-be6e-4292-940c-941e6e6873af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:45.970328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a0df3-be6e-4292-940c-941e6e6873af None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:45.970609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a0df3-be6e-4292-940c-941e6e6873af None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:45.970884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a0df3-be6e-4292-940c-941e6e6873af None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/groups/31d7b063f5f44104891f1babd3641dbc/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:45.972837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a0df3-be6e-4292-940c-941e6e6873af None admin] RBAC: Authorizing `identity:create_grant(project_id=a747fd9ec2214dcc9ddec8091b57baea, group_id=31d7b063f5f44104891f1babd3641dbc, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:45.974405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a0df3-be6e-4292-940c-941e6e6873af None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:46.081920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 198/387] 10.222.0.83 () {58 vars in 1442 bytes} [Wed Aug 24 11:04:45 2022] PUT /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/groups/31d7b063f5f44104891f1babd3641dbc/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:46.312157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48496-21ff-400a-a960-3d54108099f8 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:46.312695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48496-21ff-400a-a960-3d54108099f8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:46.312695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48496-21ff-400a-a960-3d54108099f8 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:46.315066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 190/388] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:46.323041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894f968-8a14-442f-8be3-23b118b9e633 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:46.323041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894f968-8a14-442f-8be3-23b118b9e633 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:46.323041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894f968-8a14-442f-8be3-23b118b9e633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:46.350455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0894f968-8a14-442f-8be3-23b118b9e633 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:46.352037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0894f968-8a14-442f-8be3-23b118b9e633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:46.359453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3043b-5442-44a2-9bef-5750b98a3536 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:46.359769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3043b-5442-44a2-9bef-5750b98a3536 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:46.360013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3043b-5442-44a2-9bef-5750b98a3536 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:46.362371 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 191/389] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:46.373985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195193e6-3b7f-4fba-9a90-9f8bf34d1fed None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:46.373985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195193e6-3b7f-4fba-9a90-9f8bf34d1fed None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:46.373985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195193e6-3b7f-4fba-9a90-9f8bf34d1fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:46.397473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-195193e6-3b7f-4fba-9a90-9f8bf34d1fed None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:46.398825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-195193e6-3b7f-4fba-9a90-9f8bf34d1fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:46.476821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 199/390] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:46.481210 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 192/391] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:46.766289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894f968-8a14-442f-8be3-23b118b9e633 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:46.865531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:46.894951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894f968-8a14-442f-8be3-23b118b9e633 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:46.947320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46f3b4-b4e8-4d8b-b2cb-3e127694b3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:46.947984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46f3b4-b4e8-4d8b-b2cb-3e127694b3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:46.948269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46f3b4-b4e8-4d8b-b2cb-3e127694b3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:46.948546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46f3b4-b4e8-4d8b-b2cb-3e127694b3a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:46.960229 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46f3b4-b4e8-4d8b-b2cb-3e127694b3a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-de46f3b4-b4e8-4d8b-b2cb-3e127694b3a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:46.965126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:46.966129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:46.967121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 200/392] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:46.972523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195193e6-3b7f-4fba-9a90-9f8bf34d1fed None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:47.034381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:47.108877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195193e6-3b7f-4fba-9a90-9f8bf34d1fed None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:47.151374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46f3b4-b4e8-4d8b-b2cb-3e127694b3a1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:47.168975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1e0a6-8be8-4ef9-972d-12390e3985f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:47.170039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1e0a6-8be8-4ef9-972d-12390e3985f2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.170700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1e0a6-8be8-4ef9-972d-12390e3985f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.171402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1e0a6-8be8-4ef9-972d-12390e3985f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.172305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1e0a6-8be8-4ef9-972d-12390e3985f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:47.173013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1e0a6-8be8-4ef9-972d-12390e3985f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:47.177036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1e0a6-8be8-4ef9-972d-12390e3985f2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:47.202392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 193/393] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:47.214213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1e0a6-8be8-4ef9-972d-12390e3985f2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:47.384686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46f3b4-b4e8-4d8b-b2cb-3e127694b3a1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:47.409800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1e0a6-8be8-4ef9-972d-12390e3985f2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:47.419518 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:47.454336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746d467-2887-4d68-8952-6eb2966342e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:47.455423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d467-2887-4d68-8952-6eb2966342e9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.455706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d467-2887-4d68-8952-6eb2966342e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.456020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d467-2887-4d68-8952-6eb2966342e9 None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.457846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d467-2887-4d68-8952-6eb2966342e9 None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:47.459505 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d467-2887-4d68-8952-6eb2966342e9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:47.466655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 194/394] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:47 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:47.468716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746d467-2887-4d68-8952-6eb2966342e9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:47.504011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4dc91-8ec2-4a18-aa1f-8d82afe788e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:47.504975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4dc91-8ec2-4a18-aa1f-8d82afe788e9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.505255 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4dc91-8ec2-4a18-aa1f-8d82afe788e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.510297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4dc91-8ec2-4a18-aa1f-8d82afe788e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.515659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:47.540558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4dc91-8ec2-4a18-aa1f-8d82afe788e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7ca4dc91-8ec2-4a18-aa1f-8d82afe788e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:47.548722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:47.550296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:47.551545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:47.555398 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 201/395] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:47.556973 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4dc91-8ec2-4a18-aa1f-8d82afe788e9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:47.681527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746d467-2887-4d68-8952-6eb2966342e9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:47.703726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4dc91-8ec2-4a18-aa1f-8d82afe788e9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:47.720098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81076d-e7b0-48d8-8995-6323d2b67731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:47.720740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81076d-e7b0-48d8-8995-6323d2b67731 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.721025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81076d-e7b0-48d8-8995-6323d2b67731 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.721303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81076d-e7b0-48d8-8995-6323d2b67731 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.722504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81076d-e7b0-48d8-8995-6323d2b67731 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:47.723962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81076d-e7b0-48d8-8995-6323d2b67731 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:47.725949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 195/396] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:47 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:47.727214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1332d27-0258-4a9f-a627-cfcf0da1845a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.727499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1332d27-0258-4a9f-a627-cfcf0da1845a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.727774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1332d27-0258-4a9f-a627-cfcf0da1845a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.729070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 196/397] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:47.734431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31a9d6-5d26-4a64-8634-ee2824da8b07 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.734782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31a9d6-5d26-4a64-8634-ee2824da8b07 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.735127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31a9d6-5d26-4a64-8634-ee2824da8b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.752961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fae8c1-f8a5-4e31-81da-04d3ef35252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:47.753581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fae8c1-f8a5-4e31-81da-04d3ef35252b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.753924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fae8c1-f8a5-4e31-81da-04d3ef35252b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.754208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fae8c1-f8a5-4e31-81da-04d3ef35252b None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.754846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7e31a9d6-5d26-4a64-8634-ee2824da8b07 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:47.755017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fae8c1-f8a5-4e31-81da-04d3ef35252b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:47.755323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fae8c1-f8a5-4e31-81da-04d3ef35252b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:47.756242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7e31a9d6-5d26-4a64-8634-ee2824da8b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:47.756884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fae8c1-f8a5-4e31-81da-04d3ef35252b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:47.772519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 202/398] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:47.773493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fae8c1-f8a5-4e31-81da-04d3ef35252b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:47.796820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:47.831313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 197/399] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:47.832270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31a9d6-5d26-4a64-8634-ee2824da8b07 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:47.851020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:47.899772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fae8c1-f8a5-4e31-81da-04d3ef35252b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:47.937150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31a9d6-5d26-4a64-8634-ee2824da8b07 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:47.939385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a2053-cb4e-412f-b1de-d42e1f03bf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:47.940326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a2053-cb4e-412f-b1de-d42e1f03bf22 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.940882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a2053-cb4e-412f-b1de-d42e1f03bf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.941368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a2053-cb4e-412f-b1de-d42e1f03bf22 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507/groups/31d7b063f5f44104891f1babd3641dbc/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.944457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a2053-cb4e-412f-b1de-d42e1f03bf22 None admin] RBAC: Authorizing `identity:create_grant(project_id=858cb936469e4382b567530855fa7507, group_id=31d7b063f5f44104891f1babd3641dbc, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:47.946380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a2053-cb4e-412f-b1de-d42e1f03bf22 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:47.979267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd871c9-1292-406b-b5b2-833d5793a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:47.980156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd871c9-1292-406b-b5b2-833d5793a4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.980473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd871c9-1292-406b-b5b2-833d5793a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.980747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd871c9-1292-406b-b5b2-833d5793a4d8 None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.981916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd871c9-1292-406b-b5b2-833d5793a4d8 None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:47.983354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd871c9-1292-406b-b5b2-833d5793a4d8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:47.985119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 198/400] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:47 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:47.989557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6389979-82c5-4d6f-8907-c41e577e0a41 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:47.990148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6389979-82c5-4d6f-8907-c41e577e0a41 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:47.990636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6389979-82c5-4d6f-8907-c41e577e0a41 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:47.992097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 199/401] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:47.992671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6389979-82c5-4d6f-8907-c41e577e0a41 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:48.053987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:48.080288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 203/402] 10.222.0.83 () {58 vars in 1442 bytes} [Wed Aug 24 11:04:47 2022] PUT /identity/v3/projects/858cb936469e4382b567530855fa7507/groups/31d7b063f5f44104891f1babd3641dbc/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:48.088005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129394b8-99a9-46bd-b8de-dff3e1090f74 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:48.088656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129394b8-99a9-46bd-b8de-dff3e1090f74 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:48.089129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129394b8-99a9-46bd-b8de-dff3e1090f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:48.123769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-129394b8-99a9-46bd-b8de-dff3e1090f74 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:48.125513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-129394b8-99a9-46bd-b8de-dff3e1090f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:48.131762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6389979-82c5-4d6f-8907-c41e577e0a41 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:48.189508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcf6ee-6505-4899-ad5f-52489e3a315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:48.190984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcf6ee-6505-4899-ad5f-52489e3a315b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:48.191505 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcf6ee-6505-4899-ad5f-52489e3a315b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:48.191924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcf6ee-6505-4899-ad5f-52489e3a315b None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:48.193419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcf6ee-6505-4899-ad5f-52489e3a315b None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:48.195559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcf6ee-6505-4899-ad5f-52489e3a315b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:48.205354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 200/403] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:47 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:48.206664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcf6ee-6505-4899-ad5f-52489e3a315b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:48.242795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 204/404] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:48.244326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129394b8-99a9-46bd-b8de-dff3e1090f74 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:48.335249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:48.404098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcf6ee-6505-4899-ad5f-52489e3a315b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:48.418687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129394b8-99a9-46bd-b8de-dff3e1090f74 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:48.425912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:48.468555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd011f4-bab2-4e1e-922e-4e7c676901b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:48.469150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd011f4-bab2-4e1e-922e-4e7c676901b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:48.469469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd011f4-bab2-4e1e-922e-4e7c676901b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:48.469743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd011f4-bab2-4e1e-922e-4e7c676901b6 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/groups/31d7b063f5f44104891f1babd3641dbc/roles/e08fcbbf38b344f48f71ed9ceb314293` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:48.473912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd011f4-bab2-4e1e-922e-4e7c676901b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a747fd9ec2214dcc9ddec8091b57baea, group_id=31d7b063f5f44104891f1babd3641dbc, role_id=e08fcbbf38b344f48f71ed9ceb314293)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:48.475515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd011f4-bab2-4e1e-922e-4e7c676901b6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:48.479974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe4331-46a2-4078-9163-a2e68c7999a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:48.480961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe4331-46a2-4078-9163-a2e68c7999a5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:48.481305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe4331-46a2-4078-9163-a2e68c7999a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:48.481625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe4331-46a2-4078-9163-a2e68c7999a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:48.497312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe4331-46a2-4078-9163-a2e68c7999a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-05fe4331-46a2-4078-9163-a2e68c7999a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:48.502739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:48.503737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:48.504669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 201/405] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:48.510302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe4331-46a2-4078-9163-a2e68c7999a5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:48.577114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 205/406] 10.222.0.83 () {58 vars in 1442 bytes} [Wed Aug 24 11:04:48 2022] PUT /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea/groups/31d7b063f5f44104891f1babd3641dbc/roles/e08fcbbf38b344f48f71ed9ceb314293 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:48.578917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd011f4-bab2-4e1e-922e-4e7c676901b6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:48.671884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe4331-46a2-4078-9163-a2e68c7999a5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:48.741893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94836703-450f-4107-abf3-32ddd44d360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:48.742735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94836703-450f-4107-abf3-32ddd44d360d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:48.743023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94836703-450f-4107-abf3-32ddd44d360d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:48.743731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94836703-450f-4107-abf3-32ddd44d360d None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:48.744573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94836703-450f-4107-abf3-32ddd44d360d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:48.745111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94836703-450f-4107-abf3-32ddd44d360d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:48.747502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94836703-450f-4107-abf3-32ddd44d360d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:48.761978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 202/407] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:48.763732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886e4d1-8feb-4b64-a0fe-7d529bcf3231 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:48.764210 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886e4d1-8feb-4b64-a0fe-7d529bcf3231 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:48.764672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886e4d1-8feb-4b64-a0fe-7d529bcf3231 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:48.767739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 203/408] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:48.769018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886e4d1-8feb-4b64-a0fe-7d529bcf3231 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:48.771323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd011f4-bab2-4e1e-922e-4e7c676901b6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:48.782848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:48.817750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:48.825368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7c628-0b74-40b6-8be9-8c77930f1515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:48.826854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7c628-0b74-40b6-8be9-8c77930f1515 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:48.827185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7c628-0b74-40b6-8be9-8c77930f1515 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:48.827462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7c628-0b74-40b6-8be9-8c77930f1515 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:48.847300 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7c628-0b74-40b6-8be9-8c77930f1515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7da7c628-0b74-40b6-8be9-8c77930f1515 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:48.853523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:48.854544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:48.855424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 206/409] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:04:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:48.858002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a0d79-e945-4f55-aaa8-bb939aa8769b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:48.858161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a0d79-e945-4f55-aaa8-bb939aa8769b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:48.858502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a0d79-e945-4f55-aaa8-bb939aa8769b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:48.879696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b95a0d79-e945-4f55-aaa8-bb939aa8769b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:48.880920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b95a0d79-e945-4f55-aaa8-bb939aa8769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:48.926114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886e4d1-8feb-4b64-a0fe-7d529bcf3231 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:48.966832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 207/410] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:48.967526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a0d79-e945-4f55-aaa8-bb939aa8769b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:48.969131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2538458-3fb1-4f25-94d0-0812acb0f858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:48.969836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538458-3fb1-4f25-94d0-0812acb0f858 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:48.970211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538458-3fb1-4f25-94d0-0812acb0f858 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:48.970656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538458-3fb1-4f25-94d0-0812acb0f858 None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:48.972040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2538458-3fb1-4f25-94d0-0812acb0f858 None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:48.973714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2538458-3fb1-4f25-94d0-0812acb0f858 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:48.975940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 204/411] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:48 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:48.987076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2538458-3fb1-4f25-94d0-0812acb0f858 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:49.106590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:49.114114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a0d79-e945-4f55-aaa8-bb939aa8769b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:49.123976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:49.129000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2538458-3fb1-4f25-94d0-0812acb0f858 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:49.155381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18069fa7-1857-43a1-9f53-f5d40ac97d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:49.156400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18069fa7-1857-43a1-9f53-f5d40ac97d60 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:49.156881 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18069fa7-1857-43a1-9f53-f5d40ac97d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:49.157388 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18069fa7-1857-43a1-9f53-f5d40ac97d60 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:49.158306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18069fa7-1857-43a1-9f53-f5d40ac97d60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:49.158815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18069fa7-1857-43a1-9f53-f5d40ac97d60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:49.160420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18069fa7-1857-43a1-9f53-f5d40ac97d60 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:49.166590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87399be-cee9-4c0b-a600-7220248b52a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:49.167262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87399be-cee9-4c0b-a600-7220248b52a5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:49.167583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87399be-cee9-4c0b-a600-7220248b52a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:49.167864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87399be-cee9-4c0b-a600-7220248b52a5 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:49.169117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87399be-cee9-4c0b-a600-7220248b52a5 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:49.170767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87399be-cee9-4c0b-a600-7220248b52a5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:49.171289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 208/412] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:49.178050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 205/413] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:48 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:49.179770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18069fa7-1857-43a1-9f53-f5d40ac97d60 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:49.180129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87399be-cee9-4c0b-a600-7220248b52a5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:49.301915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18069fa7-1857-43a1-9f53-f5d40ac97d60 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:49.302307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87399be-cee9-4c0b-a600-7220248b52a5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:49.342573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808d5a-1a19-4b79-abdf-e4073ca3d919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:49.343307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808d5a-1a19-4b79-abdf-e4073ca3d919 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:49.343592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808d5a-1a19-4b79-abdf-e4073ca3d919 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:49.343867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808d5a-1a19-4b79-abdf-e4073ca3d919 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507/groups/31d7b063f5f44104891f1babd3641dbc/roles/e08fcbbf38b344f48f71ed9ceb314293` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:49.346290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808d5a-1a19-4b79-abdf-e4073ca3d919 None admin] RBAC: Authorizing `identity:create_grant(project_id=858cb936469e4382b567530855fa7507, group_id=31d7b063f5f44104891f1babd3641dbc, role_id=e08fcbbf38b344f48f71ed9ceb314293)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:49.347784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808d5a-1a19-4b79-abdf-e4073ca3d919 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:49.351349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f72bd-0ffe-41e5-af1f-cadf2a783eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:49.352009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f72bd-0ffe-41e5-af1f-cadf2a783eac None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:49.352370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f72bd-0ffe-41e5-af1f-cadf2a783eac None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:49.352571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f72bd-0ffe-41e5-af1f-cadf2a783eac None admin] PATH_INFO: `/v3/groups/33851d52f05144f384cd05645ac63334` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:49.354697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f72bd-0ffe-41e5-af1f-cadf2a783eac None admin] RBAC: Authorizing `identity:get_group(group_id=33851d52f05144f384cd05645ac63334)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:49.356497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f72bd-0ffe-41e5-af1f-cadf2a783eac None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:49.358532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 209/414] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:49 2022] GET /identity/v3/groups/33851d52f05144f384cd05645ac63334 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:49.365118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f72bd-0ffe-41e5-af1f-cadf2a783eac None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:49.399693 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:49.486850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f72bd-0ffe-41e5-af1f-cadf2a783eac None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:49.524042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 206/415] 10.222.0.83 () {58 vars in 1442 bytes} [Wed Aug 24 11:04:49 2022] PUT /identity/v3/projects/858cb936469e4382b567530855fa7507/groups/31d7b063f5f44104891f1babd3641dbc/roles/e08fcbbf38b344f48f71ed9ceb314293 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:04:49.524905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808d5a-1a19-4b79-abdf-e4073ca3d919 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:49.537788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12abb5-f69c-47e3-b9ef-6b1758392ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:49.540123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12abb5-f69c-47e3-b9ef-6b1758392ddc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:49.540730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12abb5-f69c-47e3-b9ef-6b1758392ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:49.541426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12abb5-f69c-47e3-b9ef-6b1758392ddc None admin] PATH_INFO: `/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:49.543199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12abb5-f69c-47e3-b9ef-6b1758392ddc None admin] RBAC: Authorizing `identity:get_project(project_id=17f9dd8b207b49df9eb8e79eece61c9b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:49.545786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12abb5-f69c-47e3-b9ef-6b1758392ddc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:49.549422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 210/416] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:49 2022] GET /identity/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:49.563291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12abb5-f69c-47e3-b9ef-6b1758392ddc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:49.588914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:49.774028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:49.775893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808d5a-1a19-4b79-abdf-e4073ca3d919 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:49.784333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12abb5-f69c-47e3-b9ef-6b1758392ddc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:49.838933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efd4f4-4175-47f1-be09-af6bb3063856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:49.839838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efd4f4-4175-47f1-be09-af6bb3063856 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:49.840224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efd4f4-4175-47f1-be09-af6bb3063856 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:49.840560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e1e6d-2a0f-4c77-ab82-02683d54b83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:49.840687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efd4f4-4175-47f1-be09-af6bb3063856 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:49.841700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e1e6d-2a0f-4c77-ab82-02683d54b83e None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:49.842166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e1e6d-2a0f-4c77-ab82-02683d54b83e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:49.842658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e1e6d-2a0f-4c77-ab82-02683d54b83e None admin] PATH_INFO: `/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b/groups/33851d52f05144f384cd05645ac63334/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:49.850579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e1e6d-2a0f-4c77-ab82-02683d54b83e None admin] RBAC: Authorizing `identity:create_grant(project_id=17f9dd8b207b49df9eb8e79eece61c9b, group_id=33851d52f05144f384cd05645ac63334, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:49.855147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e1e6d-2a0f-4c77-ab82-02683d54b83e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:49.872094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efd4f4-4175-47f1-be09-af6bb3063856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-35efd4f4-4175-47f1-be09-af6bb3063856 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:49.879164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:49.880170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:49.880978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 207/417] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:49.886827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efd4f4-4175-47f1-be09-af6bb3063856 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:49.976628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 211/418] 10.222.0.83 () {58 vars in 1442 bytes} [Wed Aug 24 11:04:49 2022] PUT /identity/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b/groups/33851d52f05144f384cd05645ac63334/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:04:50.032408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:50.056963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efd4f4-4175-47f1-be09-af6bb3063856 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:50.109853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395e822-4ab2-44ad-9115-14ad8b214ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:50.111444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395e822-4ab2-44ad-9115-14ad8b214ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:50.111962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395e822-4ab2-44ad-9115-14ad8b214ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:50.112462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395e822-4ab2-44ad-9115-14ad8b214ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:50.113358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395e822-4ab2-44ad-9115-14ad8b214ab1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:50.113897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395e822-4ab2-44ad-9115-14ad8b214ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:50.115647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395e822-4ab2-44ad-9115-14ad8b214ab1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:50.126681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 208/419] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:50.133933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e1e6d-2a0f-4c77-ab82-02683d54b83e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:50.254889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e1e6d-2a0f-4c77-ab82-02683d54b83e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:50.300404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6cb2d-5914-4017-8151-b9fb310e9e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:50.301417 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6cb2d-5914-4017-8151-b9fb310e9e95 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:50.301924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6cb2d-5914-4017-8151-b9fb310e9e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:50.302435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6cb2d-5914-4017-8151-b9fb310e9e95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:50.318243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f357b-e1c2-4ab9-ae95-c51a268ea0d9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:50.318551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f357b-e1c2-4ab9-ae95-c51a268ea0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:50.318799 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f357b-e1c2-4ab9-ae95-c51a268ea0d9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:50.325532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 209/420] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:04:50.331283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c616-a87c-4866-ad8a-10640d1c3e11 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:50.331842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c616-a87c-4866-ad8a-10640d1c3e11 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:50.332311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c616-a87c-4866-ad8a-10640d1c3e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:50.342209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:50.352086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1112c616-a87c-4866-ad8a-10640d1c3e11 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:50.357906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1112c616-a87c-4866-ad8a-10640d1c3e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:50.448645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6cb2d-5914-4017-8151-b9fb310e9e95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:50.453389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6cb2d-5914-4017-8151-b9fb310e9e95 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:50.478024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 210/421] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:50.576806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 212/422] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:50.582222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112c616-a87c-4866-ad8a-10640d1c3e11 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:50.707807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:50.739755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112c616-a87c-4866-ad8a-10640d1c3e11 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:50.784567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb7bd4-72a0-408f-a0fa-ffb934cd6f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:50.785211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb7bd4-72a0-408f-a0fa-ffb934cd6f34 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:50.785495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb7bd4-72a0-408f-a0fa-ffb934cd6f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:50.785774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb7bd4-72a0-408f-a0fa-ffb934cd6f34 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:50.786950 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb7bd4-72a0-408f-a0fa-ffb934cd6f34 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:50.788406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb7bd4-72a0-408f-a0fa-ffb934cd6f34 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:50.794232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 211/423] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:50 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:50.795948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6cb2d-5914-4017-8151-b9fb310e9e95 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:50.896810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb7bd4-72a0-408f-a0fa-ffb934cd6f34 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:50.904094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:50.953120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6cb2d-5914-4017-8151-b9fb310e9e95 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:50.994207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100460e-795f-4e73-9df3-5d448eae6ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:50.994966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100460e-795f-4e73-9df3-5d448eae6ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:50.995240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100460e-795f-4e73-9df3-5d448eae6ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:50.995525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100460e-795f-4e73-9df3-5d448eae6ed3 None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:50.999371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100460e-795f-4e73-9df3-5d448eae6ed3 None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:51.001175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100460e-795f-4e73-9df3-5d448eae6ed3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:51.005421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 213/424] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:50 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:51.013001 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100460e-795f-4e73-9df3-5d448eae6ed3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:51.129849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb7bd4-72a0-408f-a0fa-ffb934cd6f34 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:51.179767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f48f4b-3813-47b7-8169-766ef5646051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:51.180319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f48f4b-3813-47b7-8169-766ef5646051 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:51.180599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f48f4b-3813-47b7-8169-766ef5646051 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:51.181470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f48f4b-3813-47b7-8169-766ef5646051 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:51.197406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100460e-795f-4e73-9df3-5d448eae6ed3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:51.198515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f48f4b-3813-47b7-8169-766ef5646051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-93f48f4b-3813-47b7-8169-766ef5646051 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:51.205530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:51.206506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:04:51.207279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:51.207874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 212/425] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:51.212172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0162c30-81f8-4707-a1f6-90ab0533af73 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:51.212462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0162c30-81f8-4707-a1f6-90ab0533af73 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:51.212740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0162c30-81f8-4707-a1f6-90ab0533af73 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:51.214960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 213/426] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:51.215691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0162c30-81f8-4707-a1f6-90ab0533af73 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:51.231153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:51.246256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:51.248741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af6baf-2227-4ed2-970c-23ff11131e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:51.249367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af6baf-2227-4ed2-970c-23ff11131e71 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:51.249818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af6baf-2227-4ed2-970c-23ff11131e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:51.250135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af6baf-2227-4ed2-970c-23ff11131e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:51.250955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af6baf-2227-4ed2-970c-23ff11131e71 None admin] RBAC: Adding query filter params (group.id=31d7b063f5f44104891f1babd3641dbc, role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:51.251643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af6baf-2227-4ed2-970c-23ff11131e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:51.253440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af6baf-2227-4ed2-970c-23ff11131e71 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:51.265993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 214/427] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:51 2022] GET /identity/v3/role_assignments?group.id=31d7b063f5f44104891f1babd3641dbc&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=799b2f3554434281bf10b2602fbbc09a => generated 611 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:51.267924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae23250-f33e-4460-a67c-7f75c1c6494f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:51.268536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae23250-f33e-4460-a67c-7f75c1c6494f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:51.269017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae23250-f33e-4460-a67c-7f75c1c6494f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:51.311421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4ae23250-f33e-4460-a67c-7f75c1c6494f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:51.315503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4ae23250-f33e-4460-a67c-7f75c1c6494f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:51.416883 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0162c30-81f8-4707-a1f6-90ab0533af73 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:51.461767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 215/428] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:51.479708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd77055-cfe5-47f9-9602-7482985857ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:51.480373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd77055-cfe5-47f9-9602-7482985857ee None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:51.480693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd77055-cfe5-47f9-9602-7482985857ee None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:51.480979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd77055-cfe5-47f9-9602-7482985857ee None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:51.481667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd77055-cfe5-47f9-9602-7482985857ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:51.482064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd77055-cfe5-47f9-9602-7482985857ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:51.483569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd77055-cfe5-47f9-9602-7482985857ee None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:51.493605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 214/429] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:04:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:51.499264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae23250-f33e-4460-a67c-7f75c1c6494f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:51.623743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae23250-f33e-4460-a67c-7f75c1c6494f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:51.645011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:51.668505 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780712bb-81dc-4226-9fa3-0fe772b533a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:51.669139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780712bb-81dc-4226-9fa3-0fe772b533a7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:51.669435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780712bb-81dc-4226-9fa3-0fe772b533a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:51.669705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780712bb-81dc-4226-9fa3-0fe772b533a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:51.748252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780712bb-81dc-4226-9fa3-0fe772b533a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:51.749647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780712bb-81dc-4226-9fa3-0fe772b533a7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:51.777259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd77055-cfe5-47f9-9602-7482985857ee None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:51.879976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 216/430] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:51.882444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:51.887468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780712bb-81dc-4226-9fa3-0fe772b533a7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:51.908275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:51.909282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd77055-cfe5-47f9-9602-7482985857ee None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:51.964393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff904f-3db8-4e22-837b-257edc1590ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:51.965457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff904f-3db8-4e22-837b-257edc1590ee None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:51.966161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff904f-3db8-4e22-837b-257edc1590ee None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:51.966627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff904f-3db8-4e22-837b-257edc1590ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:51.982972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff904f-3db8-4e22-837b-257edc1590ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-70ff904f-3db8-4e22-837b-257edc1590ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:51.989017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:51.990006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:51.990834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 215/431] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:51.992279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696dd58-da0c-46d8-a640-a5f82235a99f None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:51.998171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696dd58-da0c-46d8-a640-a5f82235a99f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:51.998171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696dd58-da0c-46d8-a640-a5f82235a99f None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:51.998171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 216/432] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:51.999823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696dd58-da0c-46d8-a640-a5f82235a99f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:52.023834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780712bb-81dc-4226-9fa3-0fe772b533a7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:52.068462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4591de5-337e-4917-93cb-6ad2583733dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:52.069124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4591de5-337e-4917-93cb-6ad2583733dc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.071031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4591de5-337e-4917-93cb-6ad2583733dc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.071031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4591de5-337e-4917-93cb-6ad2583733dc None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.071031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4591de5-337e-4917-93cb-6ad2583733dc None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:52.072369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4591de5-337e-4917-93cb-6ad2583733dc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:52.076306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 217/433] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:51 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:52.076306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3783f64-c572-424b-b98a-fd785bc3795b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.076306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3783f64-c572-424b-b98a-fd785bc3795b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.076306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3783f64-c572-424b-b98a-fd785bc3795b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.097235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b3783f64-c572-424b-b98a-fd785bc3795b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:52.098399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b3783f64-c572-424b-b98a-fd785bc3795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:52.148836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696dd58-da0c-46d8-a640-a5f82235a99f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:52.174277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 218/434] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:52.175748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3783f64-c572-424b-b98a-fd785bc3795b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:52.192188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:52.201443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c04c5d-3c68-4aa6-a2ab-931067263d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:52.202126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:52.203156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c04c5d-3c68-4aa6-a2ab-931067263d15 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.204700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c04c5d-3c68-4aa6-a2ab-931067263d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.204700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c04c5d-3c68-4aa6-a2ab-931067263d15 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.205479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c04c5d-3c68-4aa6-a2ab-931067263d15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:52.205957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c04c5d-3c68-4aa6-a2ab-931067263d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:52.207620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c04c5d-3c68-4aa6-a2ab-931067263d15 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:52.218615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 217/435] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:52.225248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c04c5d-3c68-4aa6-a2ab-931067263d15 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:52.312791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3783f64-c572-424b-b98a-fd785bc3795b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:52.353212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bd4b8-931a-4d5c-80b5-8685f73a761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:52.354460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bd4b8-931a-4d5c-80b5-8685f73a761e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.355146 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bd4b8-931a-4d5c-80b5-8685f73a761e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.355591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bd4b8-931a-4d5c-80b5-8685f73a761e None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.357492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bd4b8-931a-4d5c-80b5-8685f73a761e None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:52.358710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c04c5d-3c68-4aa6-a2ab-931067263d15 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:52.359331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bd4b8-931a-4d5c-80b5-8685f73a761e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:52.362884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 219/436] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:52 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:52.364228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745711c-2586-498a-bdec-5c552cb73d7b None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.364664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745711c-2586-498a-bdec-5c552cb73d7b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.365084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745711c-2586-498a-bdec-5c552cb73d7b None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.366534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 220/437] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:52.368592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745711c-2586-498a-bdec-5c552cb73d7b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:52.404216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1dc6b-cbbc-49c5-ac38-d1b97354cd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:52.405219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1dc6b-cbbc-49c5-ac38-d1b97354cd0c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.405740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1dc6b-cbbc-49c5-ac38-d1b97354cd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.406053 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1dc6b-cbbc-49c5-ac38-d1b97354cd0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.465298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:52.474706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:52.484342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745711c-2586-498a-bdec-5c552cb73d7b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:52.485179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1dc6b-cbbc-49c5-ac38-d1b97354cd0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:52.487231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1dc6b-cbbc-49c5-ac38-d1b97354cd0c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:52.521638 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d15f4-d714-455e-9230-64526e6b06be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:52.522277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d15f4-d714-455e-9230-64526e6b06be None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.522557 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d15f4-d714-455e-9230-64526e6b06be None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.522831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d15f4-d714-455e-9230-64526e6b06be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.523569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d15f4-d714-455e-9230-64526e6b06be None admin] RBAC: Adding query filter params (group.id=31d7b063f5f44104891f1babd3641dbc, role.id=799b2f3554434281bf10b2602fbbc09a, scope.project.id=858cb936469e4382b567530855fa7507) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:52.523878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d15f4-d714-455e-9230-64526e6b06be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:52.525323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d15f4-d714-455e-9230-64526e6b06be None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:52.531465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 221/438] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:52 2022] GET /identity/v3/role_assignments?group.id=31d7b063f5f44104891f1babd3641dbc&scope.project.id=858cb936469e4382b567530855fa7507&role.id=799b2f3554434281bf10b2602fbbc09a => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:52.533232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8407676-6b81-4529-9edd-19c45c4733cf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.533513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8407676-6b81-4529-9edd-19c45c4733cf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.533781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8407676-6b81-4529-9edd-19c45c4733cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.553643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e8407676-6b81-4529-9edd-19c45c4733cf None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:52.554778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e8407676-6b81-4529-9edd-19c45c4733cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:52.609333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 218/439] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:52.611148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1dc6b-cbbc-49c5-ac38-d1b97354cd0c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:52.639147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 222/440] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:52.639869 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8407676-6b81-4529-9edd-19c45c4733cf None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:52.754237 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:52.765394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:52.777763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1dc6b-cbbc-49c5-ac38-d1b97354cd0c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:52.784776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8407676-6b81-4529-9edd-19c45c4733cf None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:52.816983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cad8a-7517-4e50-8084-98c319d1ccbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:52.817539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cad8a-7517-4e50-8084-98c319d1ccbf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.817816 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cad8a-7517-4e50-8084-98c319d1ccbf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.818112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cad8a-7517-4e50-8084-98c319d1ccbf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.825455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1759d19-f6b4-481a-b71a-c5eb60b51c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:52.826126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1759d19-f6b4-481a-b71a-c5eb60b51c17 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.826406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1759d19-f6b4-481a-b71a-c5eb60b51c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.826682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1759d19-f6b4-481a-b71a-c5eb60b51c17 None admin] PATH_INFO: `/v3/projects/a747fd9ec2214dcc9ddec8091b57baea` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.827815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1759d19-f6b4-481a-b71a-c5eb60b51c17 None admin] RBAC: Authorizing `identity:get_project(project_id=a747fd9ec2214dcc9ddec8091b57baea)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:52.829358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1759d19-f6b4-481a-b71a-c5eb60b51c17 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:52.829532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cad8a-7517-4e50-8084-98c319d1ccbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:52.831239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 223/441] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:52 2022] GET /identity/v3/projects/a747fd9ec2214dcc9ddec8091b57baea => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-1c4cad8a-7517-4e50-8084-98c319d1ccbf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:52.834370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:52.835569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:52.836571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:52.837381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1759d19-f6b4-481a-b71a-c5eb60b51c17 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:52.840525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 219/442] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:52.841845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cad8a-7517-4e50-8084-98c319d1ccbf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:52.948129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cad8a-7517-4e50-8084-98c319d1ccbf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:52.953350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1759d19-f6b4-481a-b71a-c5eb60b51c17 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:52.983318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378fa10-5a31-41e3-bd4c-4b814a4026a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:52.983955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378fa10-5a31-41e3-bd4c-4b814a4026a4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.984239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378fa10-5a31-41e3-bd4c-4b814a4026a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.984517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378fa10-5a31-41e3-bd4c-4b814a4026a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.985263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378fa10-5a31-41e3-bd4c-4b814a4026a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:52.985555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378fa10-5a31-41e3-bd4c-4b814a4026a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:52.986938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378fa10-5a31-41e3-bd4c-4b814a4026a4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:52.988616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132d965-9b2d-4a0d-b287-e4b23ac31cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:52.989318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132d965-9b2d-4a0d-b287-e4b23ac31cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:52.989522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132d965-9b2d-4a0d-b287-e4b23ac31cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:52.989799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132d965-9b2d-4a0d-b287-e4b23ac31cc9 None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:52.990958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132d965-9b2d-4a0d-b287-e4b23ac31cc9 None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:52.992407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132d965-9b2d-4a0d-b287-e4b23ac31cc9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:52.994235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 224/443] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:52 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:52.994831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132d965-9b2d-4a0d-b287-e4b23ac31cc9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:52.995381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 220/444] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:53.000121 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378fa10-5a31-41e3-bd4c-4b814a4026a4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:53.017015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:53.017614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:53.113132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132d965-9b2d-4a0d-b287-e4b23ac31cc9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:53.128623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378fa10-5a31-41e3-bd4c-4b814a4026a4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:53.148459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308c4a5-0458-413d-b94b-dbb1991f4396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:53.149097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308c4a5-0458-413d-b94b-dbb1991f4396 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.149377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308c4a5-0458-413d-b94b-dbb1991f4396 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.149650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308c4a5-0458-413d-b94b-dbb1991f4396 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.160663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308c4a5-0458-413d-b94b-dbb1991f4396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:53.164348 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ccef6-c7ac-4b53-b673-c65f08d47988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:53.164947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ccef6-c7ac-4b53-b673-c65f08d47988 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.165224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ccef6-c7ac-4b53-b673-c65f08d47988 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.165496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ccef6-c7ac-4b53-b673-c65f08d47988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7308c4a5-0458-413d-b94b-dbb1991f4396 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:53.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:53.166749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:04:53.167698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:53.168665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ccef6-c7ac-4b53-b673-c65f08d47988 None admin] RBAC: Adding query filter params (group.id=31d7b063f5f44104891f1babd3641dbc, role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=a747fd9ec2214dcc9ddec8091b57baea) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:53.168665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ccef6-c7ac-4b53-b673-c65f08d47988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:53.168827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 225/445] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:04:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:53.168827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308c4a5-0458-413d-b94b-dbb1991f4396 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:53.172805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ccef6-c7ac-4b53-b673-c65f08d47988 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:53.180116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 221/446] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:52 2022] GET /identity/v3/role_assignments?group.id=31d7b063f5f44104891f1babd3641dbc&scope.project.id=a747fd9ec2214dcc9ddec8091b57baea&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:53.194108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ccef6-c7ac-4b53-b673-c65f08d47988 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:53.256282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:53.278382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:53.292391 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308c4a5-0458-413d-b94b-dbb1991f4396 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:53.305799 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ccef6-c7ac-4b53-b673-c65f08d47988 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:53.338866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd76ad-9c49-4568-a70a-79bf6746e863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:53.339478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd76ad-9c49-4568-a70a-79bf6746e863 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.339760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd76ad-9c49-4568-a70a-79bf6746e863 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.340030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd76ad-9c49-4568-a70a-79bf6746e863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.346075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed05d9-17cd-4b6b-9977-68b806780a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:53.346683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed05d9-17cd-4b6b-9977-68b806780a4c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.346966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed05d9-17cd-4b6b-9977-68b806780a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.347236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed05d9-17cd-4b6b-9977-68b806780a4c None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.347914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed05d9-17cd-4b6b-9977-68b806780a4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:53.348219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed05d9-17cd-4b6b-9977-68b806780a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:53.349584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed05d9-17cd-4b6b-9977-68b806780a4c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:53.359366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 222/447] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:04:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:53.365387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed05d9-17cd-4b6b-9977-68b806780a4c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:53.425346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd76ad-9c49-4568-a70a-79bf6746e863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:53.427215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd76ad-9c49-4568-a70a-79bf6746e863 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:53.492673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed05d9-17cd-4b6b-9977-68b806780a4c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:53.512281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:53.535658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a0702-3f48-4c67-8627-06ddbc6c0091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:53.536516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a0702-3f48-4c67-8627-06ddbc6c0091 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.536633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a0702-3f48-4c67-8627-06ddbc6c0091 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.536813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a0702-3f48-4c67-8627-06ddbc6c0091 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.549784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:53.550659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 226/448] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:04:53.556634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd76ad-9c49-4568-a70a-79bf6746e863 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:53.625337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a0702-3f48-4c67-8627-06ddbc6c0091 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:53.627210 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a0702-3f48-4c67-8627-06ddbc6c0091 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:53.691919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd76ad-9c49-4568-a70a-79bf6746e863 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:53.740155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76be93-b94e-410d-a8a2-3cb4c85349b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:53.740783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76be93-b94e-410d-a8a2-3cb4c85349b2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.741057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76be93-b94e-410d-a8a2-3cb4c85349b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.741353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76be93-b94e-410d-a8a2-3cb4c85349b2 None admin] PATH_INFO: `/v3/projects/858cb936469e4382b567530855fa7507` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.742832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76be93-b94e-410d-a8a2-3cb4c85349b2 None admin] RBAC: Authorizing `identity:get_project(project_id=858cb936469e4382b567530855fa7507)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:53.744473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76be93-b94e-410d-a8a2-3cb4c85349b2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:53.746128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 227/449] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:53 2022] GET /identity/v3/projects/858cb936469e4382b567530855fa7507 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:53.752256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76be93-b94e-410d-a8a2-3cb4c85349b2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:53.756982 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 223/450] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:04:53.762764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a0702-3f48-4c67-8627-06ddbc6c0091 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:53.794130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:53.801763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:53.855677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76be93-b94e-410d-a8a2-3cb4c85349b2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:53.862000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a0702-3f48-4c67-8627-06ddbc6c0091 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:53.890768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca533-ed51-4799-96c5-186222c8725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:53.891406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca533-ed51-4799-96c5-186222c8725f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.892186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca533-ed51-4799-96c5-186222c8725f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.892458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca533-ed51-4799-96c5-186222c8725f None admin] PATH_INFO: `/v3/groups/31d7b063f5f44104891f1babd3641dbc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.893578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca533-ed51-4799-96c5-186222c8725f None admin] RBAC: Authorizing `identity:get_group(group_id=31d7b063f5f44104891f1babd3641dbc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:53.895105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca533-ed51-4799-96c5-186222c8725f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:53.896617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7de74e-0355-4efc-b40d-b8269f27f103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:53.896730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 228/451] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:53 2022] GET /identity/v3/groups/31d7b063f5f44104891f1babd3641dbc => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:53.897323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7de74e-0355-4efc-b40d-b8269f27f103 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:53.897608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7de74e-0355-4efc-b40d-b8269f27f103 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:53.898044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7de74e-0355-4efc-b40d-b8269f27f103 None admin] PATH_INFO: `/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:53.899182 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7de74e-0355-4efc-b40d-b8269f27f103 None admin] RBAC: Authorizing `identity:get_project(project_id=17f9dd8b207b49df9eb8e79eece61c9b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:53.900630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7de74e-0355-4efc-b40d-b8269f27f103 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:53.902383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 224/452] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:04:53 2022] GET /identity/v3/projects/17f9dd8b207b49df9eb8e79eece61c9b => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:53.902485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca533-ed51-4799-96c5-186222c8725f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:53.908457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7de74e-0355-4efc-b40d-b8269f27f103 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:54.004467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca533-ed51-4799-96c5-186222c8725f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:54.011872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7de74e-0355-4efc-b40d-b8269f27f103 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:54.013705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:54.020328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:54.039147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedfc27-6b61-439c-9d50-6c9f07148c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:54.039698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedfc27-6b61-439c-9d50-6c9f07148c52 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:54.039979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedfc27-6b61-439c-9d50-6c9f07148c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:54.040254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedfc27-6b61-439c-9d50-6c9f07148c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:54.040994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedfc27-6b61-439c-9d50-6c9f07148c52 None admin] RBAC: Adding query filter params (group.id=31d7b063f5f44104891f1babd3641dbc, role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=858cb936469e4382b567530855fa7507) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:54.041312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedfc27-6b61-439c-9d50-6c9f07148c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:54.042790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedfc27-6b61-439c-9d50-6c9f07148c52 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:54.052546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 229/453] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:53 2022] GET /identity/v3/role_assignments?group.id=31d7b063f5f44104891f1babd3641dbc&scope.project.id=858cb936469e4382b567530855fa7507&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:54.054686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2020321-e201-49fb-973c-4dd074707d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:54.055453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2020321-e201-49fb-973c-4dd074707d63 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:54.055732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2020321-e201-49fb-973c-4dd074707d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:54.058491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2020321-e201-49fb-973c-4dd074707d63 None admin] PATH_INFO: `/v3/groups/33851d52f05144f384cd05645ac63334` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:54.058491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2020321-e201-49fb-973c-4dd074707d63 None admin] RBAC: Authorizing `identity:get_group(group_id=33851d52f05144f384cd05645ac63334)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:54.059178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2020321-e201-49fb-973c-4dd074707d63 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:54.060872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 225/454] 10.222.0.83 () {56 vars in 1180 bytes} [Wed Aug 24 11:04:53 2022] GET /identity/v3/groups/33851d52f05144f384cd05645ac63334 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:04:54.067762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedfc27-6b61-439c-9d50-6c9f07148c52 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:54.179621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedfc27-6b61-439c-9d50-6c9f07148c52 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:54.217530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e32b51-c6cb-4ead-b09d-2e1c2bec3519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:54.218178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e32b51-c6cb-4ead-b09d-2e1c2bec3519 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:54.218461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e32b51-c6cb-4ead-b09d-2e1c2bec3519 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:54.218735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e32b51-c6cb-4ead-b09d-2e1c2bec3519 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:54.219484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e32b51-c6cb-4ead-b09d-2e1c2bec3519 None admin] RBAC: Adding query filter params (group.id=33851d52f05144f384cd05645ac63334, role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=17f9dd8b207b49df9eb8e79eece61c9b) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:54.219796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e32b51-c6cb-4ead-b09d-2e1c2bec3519 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:54.221258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e32b51-c6cb-4ead-b09d-2e1c2bec3519 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:54.227503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 230/455] 10.222.0.83 () {56 vars in 1376 bytes} [Wed Aug 24 11:04:54 2022] GET /identity/v3/role_assignments?group.id=33851d52f05144f384cd05645ac63334&scope.project.id=17f9dd8b207b49df9eb8e79eece61c9b&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:56.859476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229f316-814b-4c6a-bc72-2a57892f6f45 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:56.859993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229f316-814b-4c6a-bc72-2a57892f6f45 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:56.859993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229f316-814b-4c6a-bc72-2a57892f6f45 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:56.861660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 226/456] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:56.867371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe300c-f6a2-4ea4-beec-0f51d83c5b4a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:56.867832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe300c-f6a2-4ea4-beec-0f51d83c5b4a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:56.867907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe300c-f6a2-4ea4-beec-0f51d83c5b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:56.894467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b05875-e2a3-46a7-a22e-d90f29e1d604 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:56.894773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b05875-e2a3-46a7-a22e-d90f29e1d604 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:56.894958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6cbe300c-f6a2-4ea4-beec-0f51d83c5b4a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:56.895057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b05875-e2a3-46a7-a22e-d90f29e1d604 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:56.896142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6cbe300c-f6a2-4ea4-beec-0f51d83c5b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:56.896722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 227/457] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:56.903481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854cd37-7235-4f79-903a-6e6775dfcebb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:56.903750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854cd37-7235-4f79-903a-6e6775dfcebb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:56.904029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854cd37-7235-4f79-903a-6e6775dfcebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:56.938746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1854cd37-7235-4f79-903a-6e6775dfcebb None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:56.939954 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1854cd37-7235-4f79-903a-6e6775dfcebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:56.952243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:57.012589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 231/458] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:57.014384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b87625-3f71-4af1-8e98-5676b882fac9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:57.014879 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b87625-3f71-4af1-8e98-5676b882fac9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:57.015350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b87625-3f71-4af1-8e98-5676b882fac9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:57.022092 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 232/459] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:57.023824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19763fd7-e83b-48fd-9a38-81dd38e986e8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:57.024339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19763fd7-e83b-48fd-9a38-81dd38e986e8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:57.024886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19763fd7-e83b-48fd-9a38-81dd38e986e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:57.040096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 228/460] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:57.045382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-19763fd7-e83b-48fd-9a38-81dd38e986e8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:57.047162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-19763fd7-e83b-48fd-9a38-81dd38e986e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:57.162654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 233/461] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:57.364546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854cd37-7235-4f79-903a-6e6775dfcebb None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:57.380103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e9f50-1ac3-47a0-8a95-9a986519191d None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:57.380293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e9f50-1ac3-47a0-8a95-9a986519191d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:57.380555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e9f50-1ac3-47a0-8a95-9a986519191d None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:57.381931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 234/462] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:04:57.388060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fef20-9cd3-439f-9a2d-1cab409fdb3d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:57.388340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fef20-9cd3-439f-9a2d-1cab409fdb3d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:57.388615 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fef20-9cd3-439f-9a2d-1cab409fdb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:57.421992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-808fef20-9cd3-439f-9a2d-1cab409fdb3d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:57.423704 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-808fef20-9cd3-439f-9a2d-1cab409fdb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:57.520503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854cd37-7235-4f79-903a-6e6775dfcebb None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:57.522485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 235/463] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:04:57.522937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:57.523684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fef20-9cd3-439f-9a2d-1cab409fdb3d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:57.558512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:57.575923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c34e0-45c3-467b-9093-312add2f1af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:57.577094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c34e0-45c3-467b-9093-312add2f1af6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:57.577525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c34e0-45c3-467b-9093-312add2f1af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:57.578267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c34e0-45c3-467b-9093-312add2f1af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:57.592932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c34e0-45c3-467b-9093-312add2f1af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:57.595890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c34e0-45c3-467b-9093-312add2f1af6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0f9c34e0-45c3-467b-9093-312add2f1af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:04:57.617888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:57.619030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:04:57.620136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:04:57.620136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:04:57.620136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:57.627770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 229/464] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:57.627770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c34e0-45c3-467b-9093-312add2f1af6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:57.649815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fef20-9cd3-439f-9a2d-1cab409fdb3d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:57.690241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba939373-8cc7-4f70-ab53-a7d61e7d39af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:57.690803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939373-8cc7-4f70-ab53-a7d61e7d39af None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:57.691093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939373-8cc7-4f70-ab53-a7d61e7d39af None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:57.691369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939373-8cc7-4f70-ab53-a7d61e7d39af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:57.702667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba939373-8cc7-4f70-ab53-a7d61e7d39af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:57.704178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba939373-8cc7-4f70-ab53-a7d61e7d39af None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ba939373-8cc7-4f70-ab53-a7d61e7d39af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:04:57.721864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:57.722872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:04:57.723725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:04:57.723725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:04:57.723725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:57.723725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 236/465] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:57.723725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba939373-8cc7-4f70-ab53-a7d61e7d39af None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:57.765688 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c34e0-45c3-467b-9093-312add2f1af6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:57.805744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a206f-23f3-4855-a236-f86df4cf91ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:57.806840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a206f-23f3-4855-a236-f86df4cf91ca None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:57.807319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a206f-23f3-4855-a236-f86df4cf91ca None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:57.808100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a206f-23f3-4855-a236-f86df4cf91ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:57.821165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a206f-23f3-4855-a236-f86df4cf91ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:57.823062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a206f-23f3-4855-a236-f86df4cf91ca None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:57.835183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-aa2a206f-23f3-4855-a236-f86df4cf91ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:04:57.839023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:57.840103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:04:57.840968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:04:57.840968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:04:57.840968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:57.840968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 230/466] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:57.840968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a206f-23f3-4855-a236-f86df4cf91ca None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:57.855143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:57.859345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba939373-8cc7-4f70-ab53-a7d61e7d39af None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:57.897105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ec1d9-86b6-4063-854f-e6f6dd15e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:57.897802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ec1d9-86b6-4063-854f-e6f6dd15e64a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:57.898120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ec1d9-86b6-4063-854f-e6f6dd15e64a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:57.898408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ec1d9-86b6-4063-854f-e6f6dd15e64a None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:57.899445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ec1d9-86b6-4063-854f-e6f6dd15e64a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:57.899551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ec1d9-86b6-4063-854f-e6f6dd15e64a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:57.900883 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ec1d9-86b6-4063-854f-e6f6dd15e64a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:57.920792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 237/467] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:57.922093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ec1d9-86b6-4063-854f-e6f6dd15e64a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:57.954352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a206f-23f3-4855-a236-f86df4cf91ca None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:57.992700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243913a-bebf-4726-986c-47015200ffd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:57.993313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243913a-bebf-4726-986c-47015200ffd9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:57.993596 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243913a-bebf-4726-986c-47015200ffd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:57.993897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243913a-bebf-4726-986c-47015200ffd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:57.994569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243913a-bebf-4726-986c-47015200ffd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:57.994875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243913a-bebf-4726-986c-47015200ffd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:57.996267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243913a-bebf-4726-986c-47015200ffd9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:58.009798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 231/468] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:58.010831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243913a-bebf-4726-986c-47015200ffd9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:58.049564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ec1d9-86b6-4063-854f-e6f6dd15e64a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:58.086899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:58.096989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabfa08-039f-4956-82f2-0689097c7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:58.097627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabfa08-039f-4956-82f2-0689097c7713 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:58.097946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabfa08-039f-4956-82f2-0689097c7713 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:58.098459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabfa08-039f-4956-82f2-0689097c7713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:58.106687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:58.111582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabfa08-039f-4956-82f2-0689097c7713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:58.113202 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabfa08-039f-4956-82f2-0689097c7713 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:58.124083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243913a-bebf-4726-986c-47015200ffd9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-4dabfa08-039f-4956-82f2-0689097c7713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:04:58.130408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:58.131546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:04:58.132427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:04:58.132427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:04:58.132427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:58.132427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 238/469] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:58.132650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabfa08-039f-4956-82f2-0689097c7713 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:58.169509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5646d3-30f3-4d0c-8270-658989cd0fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:58.170170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5646d3-30f3-4d0c-8270-658989cd0fed None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:58.170458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5646d3-30f3-4d0c-8270-658989cd0fed None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:58.170733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5646d3-30f3-4d0c-8270-658989cd0fed None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:58.171397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5646d3-30f3-4d0c-8270-658989cd0fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:58.171701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5646d3-30f3-4d0c-8270-658989cd0fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:58.173713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5646d3-30f3-4d0c-8270-658989cd0fed None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:58.187604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 232/470] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:58.188865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5646d3-30f3-4d0c-8270-658989cd0fed None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:58.261022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabfa08-039f-4956-82f2-0689097c7713 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:58.295317 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555d5af-7c34-458b-b7c0-d97eca87edfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:58.296029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555d5af-7c34-458b-b7c0-d97eca87edfa None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:58.296382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555d5af-7c34-458b-b7c0-d97eca87edfa None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:58.296724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555d5af-7c34-458b-b7c0-d97eca87edfa None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:58.298111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555d5af-7c34-458b-b7c0-d97eca87edfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:58.299666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555d5af-7c34-458b-b7c0-d97eca87edfa None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:58.306268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5646d3-30f3-4d0c-8270-658989cd0fed None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:58.334300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:58.341911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d68b9-3ce7-4f19-b81e-dfd7d9dae05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:58.342583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d68b9-3ce7-4f19-b81e-dfd7d9dae05d None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:58.342783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d68b9-3ce7-4f19-b81e-dfd7d9dae05d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:58.343059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d68b9-3ce7-4f19-b81e-dfd7d9dae05d None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:58.343707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d68b9-3ce7-4f19-b81e-dfd7d9dae05d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:58.345245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d68b9-3ce7-4f19-b81e-dfd7d9dae05d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:58.449972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 239/471] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:58 2022] POST /identity/v3/users => generated 248 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:58.449972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555d5af-7c34-458b-b7c0-d97eca87edfa None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:58.498867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 233/472] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:58 2022] POST /identity/v3/users => generated 246 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:58.500092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d68b9-3ce7-4f19-b81e-dfd7d9dae05d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:58.509935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:58.585110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555d5af-7c34-458b-b7c0-d97eca87edfa None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:58.638844 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0c13e-726a-41b1-8db0-96740b18c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:58.639772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0c13e-726a-41b1-8db0-96740b18c0ce None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:58.640825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0c13e-726a-41b1-8db0-96740b18c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:58.640825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0c13e-726a-41b1-8db0-96740b18c0ce None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:58.641005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec0c13e-726a-41b1-8db0-96740b18c0ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:04:58.643299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec0c13e-726a-41b1-8db0-96740b18c0ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:58.644756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec0c13e-726a-41b1-8db0-96740b18c0ce None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:58.666323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 240/473] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:04:58.667964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f72350-eb95-4c84-9a96-a35c8f27face None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:58.668491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f72350-eb95-4c84-9a96-a35c8f27face None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:58.668968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f72350-eb95-4c84-9a96-a35c8f27face None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:58.670570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 241/474] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:04:58.672715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f72350-eb95-4c84-9a96-a35c8f27face None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:58.674665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d68b9-3ce7-4f19-b81e-dfd7d9dae05d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:58.719317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ff9ee-ac2a-474a-9faa-262abfe21358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:58.720088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ff9ee-ac2a-474a-9faa-262abfe21358 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:58.720364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ff9ee-ac2a-474a-9faa-262abfe21358 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:58.720637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ff9ee-ac2a-474a-9faa-262abfe21358 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:58.721294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ff9ee-ac2a-474a-9faa-262abfe21358 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:58.723522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ff9ee-ac2a-474a-9faa-262abfe21358 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:58.801087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f72350-eb95-4c84-9a96-a35c8f27face None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:58.810140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:58.840475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d303de-5a6b-4a1f-abcc-3d8643fb03c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:58.841202 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d303de-5a6b-4a1f-abcc-3d8643fb03c0 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:58.841549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d303de-5a6b-4a1f-abcc-3d8643fb03c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:58.841914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d303de-5a6b-4a1f-abcc-3d8643fb03c0 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:58.842645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d303de-5a6b-4a1f-abcc-3d8643fb03c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:58.844160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d303de-5a6b-4a1f-abcc-3d8643fb03c0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:58.863075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 234/475] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:58 2022] POST /identity/v3/users => generated 248 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:58.864255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653ff03-a1b5-49bd-9f1e-437f221b9730 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:58.864667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653ff03-a1b5-49bd-9f1e-437f221b9730 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:58.865004 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653ff03-a1b5-49bd-9f1e-437f221b9730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:58.889640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3653ff03-a1b5-49bd-9f1e-437f221b9730 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:04:58.892232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3653ff03-a1b5-49bd-9f1e-437f221b9730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:04:58.939557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:58.985245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 235/476] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:04:58.992974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 242/477] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:04:58 2022] POST /identity/v3/users => generated 249 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:04:59.285705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653ff03-a1b5-49bd-9f1e-437f221b9730 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:59.410189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653ff03-a1b5-49bd-9f1e-437f221b9730 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:59.454064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:59.454870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:59.455159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:59.455437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:59.456128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:04:59.457508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:04:59.500092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.sql.core [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 11:04:59.500092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 11:04:59.500092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [parameters: {'id': 'c2c41fb43bba46f6a849b198740e96b4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 11:04:59.500092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81865) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result.read() Aug 24 11:04:59.517890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:04:59.518842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application yield resource Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise self.value Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application session.commit() Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 11:04:59.519793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.session.flush() Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application compat.raise_( Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 11:04:59.520646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:04:59.521519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result.read() Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:04:59.522421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [parameters: {'id': 'c2c41fb43bba46f6a849b198740e96b4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:59.523215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 11:04:59.524027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:04:59.526356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 236/478] 10.222.0.83 () {60 vars in 1131 bytes} [Wed Aug 24 11:04:59 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:59.526548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d303de-5a6b-4a1f-abcc-3d8643fb03c0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:59.682312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d303de-5a6b-4a1f-abcc-3d8643fb03c0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:04:59.728562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46b5f5-b241-4e09-93ea-d9937c791689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:04:59.729222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46b5f5-b241-4e09-93ea-d9937c791689 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:04:59.729506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46b5f5-b241-4e09-93ea-d9937c791689 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:04:59.729782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46b5f5-b241-4e09-93ea-d9937c791689 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:04:59.747443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46b5f5-b241-4e09-93ea-d9937c791689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-2f46b5f5-b241-4e09-93ea-d9937c791689 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:04:59.753397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:04:59.755519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:04:59.757538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:04:59.759067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 243/479] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:04:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:04:59.763080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:04:59.849459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:04:59.953200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49877f5-a8b5-496c-94ac-07c6e77c8500 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:00.001477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7744e4e-e461-49af-90d2-0a1dd93dd14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:00.002135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7744e4e-e461-49af-90d2-0a1dd93dd14f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:00.002381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7744e4e-e461-49af-90d2-0a1dd93dd14f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:00.002656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7744e4e-e461-49af-90d2-0a1dd93dd14f None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:00.003341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7744e4e-e461-49af-90d2-0a1dd93dd14f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:00.003645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7744e4e-e461-49af-90d2-0a1dd93dd14f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:00.005127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7744e4e-e461-49af-90d2-0a1dd93dd14f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:00.018473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 237/480] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:04:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:00.684031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d73339-81a7-4b8a-920f-131f062c7b0d None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:00.685155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d73339-81a7-4b8a-920f-131f062c7b0d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:00.685645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d73339-81a7-4b8a-920f-131f062c7b0d None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:00.687569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 244/481] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:00.693469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374651a4-6286-4635-9518-385443d53b1a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:00.694079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374651a4-6286-4635-9518-385443d53b1a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:00.694588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374651a4-6286-4635-9518-385443d53b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:00.723029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-374651a4-6286-4635-9518-385443d53b1a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:00.724633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-374651a4-6286-4635-9518-385443d53b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:00.831279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 238/482] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:01.081519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c00e5-8576-4106-b5d3-d16360458150 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.082249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c00e5-8576-4106-b5d3-d16360458150 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.082733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c00e5-8576-4106-b5d3-d16360458150 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.084587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 245/483] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:01.096084 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a12349-7a1b-4f72-bb4d-119af3f70df3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.096996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a12349-7a1b-4f72-bb4d-119af3f70df3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.097513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a12349-7a1b-4f72-bb4d-119af3f70df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.126998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e8a12349-7a1b-4f72-bb4d-119af3f70df3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:01.128656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e8a12349-7a1b-4f72-bb4d-119af3f70df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:01.158269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:01.200230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c00e5-8576-4106-b5d3-d16360458150 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:01.240938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:01.243192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 239/484] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:01.357606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c00e5-8576-4106-b5d3-d16360458150 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:01.412440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a3011-ef06-4078-b9db-744cefa19fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:01.413075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a3011-ef06-4078-b9db-744cefa19fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.413353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a3011-ef06-4078-b9db-744cefa19fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.413624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a3011-ef06-4078-b9db-744cefa19fd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.434324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a3011-ef06-4078-b9db-744cefa19fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-905a3011-ef06-4078-b9db-744cefa19fd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:01.439308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:01.440265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:01.441193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:01.441840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 246/485] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:01.446672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a12349-7a1b-4f72-bb4d-119af3f70df3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:01.523424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a3011-ef06-4078-b9db-744cefa19fd4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:01.590655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a12349-7a1b-4f72-bb4d-119af3f70df3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:01.643668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111bc20-3ef3-4538-8346-7d5c614abf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:01.644785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111bc20-3ef3-4538-8346-7d5c614abf8a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.645074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111bc20-3ef3-4538-8346-7d5c614abf8a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.645344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111bc20-3ef3-4538-8346-7d5c614abf8a None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.646154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111bc20-3ef3-4538-8346-7d5c614abf8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:01.646719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111bc20-3ef3-4538-8346-7d5c614abf8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:01.648428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111bc20-3ef3-4538-8346-7d5c614abf8a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:01.661388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 240/486] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:01.663235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:01.664401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454b3e3-4d75-4d2c-8423-38aeb41ce4a5 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.664897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454b3e3-4d75-4d2c-8423-38aeb41ce4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.665370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454b3e3-4d75-4d2c-8423-38aeb41ce4a5 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.667100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 241/487] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:01.668076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454b3e3-4d75-4d2c-8423-38aeb41ce4a5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:01.691524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a3011-ef06-4078-b9db-744cefa19fd4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:01.699512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:01.730280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce1d77-4b78-49f3-93b2-75386228a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:01.730921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce1d77-4b78-49f3-93b2-75386228a89e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.731203 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce1d77-4b78-49f3-93b2-75386228a89e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.731475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce1d77-4b78-49f3-93b2-75386228a89e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.742792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce1d77-4b78-49f3-93b2-75386228a89e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7bce1d77-4b78-49f3-93b2-75386228a89e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:01.747677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:01.748582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:01.749370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 247/488] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:01.750101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ab3e9-adc8-4cb3-acf8-5092a40e028e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.751259 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ab3e9-adc8-4cb3-acf8-5092a40e028e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.751259 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ab3e9-adc8-4cb3-acf8-5092a40e028e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.770103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-587ab3e9-adc8-4cb3-acf8-5092a40e028e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:01.771323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-587ab3e9-adc8-4cb3-acf8-5092a40e028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:01.799538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454b3e3-4d75-4d2c-8423-38aeb41ce4a5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:01.841938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1f59f-a623-4a45-868d-f480b1885785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:01.842967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1f59f-a623-4a45-868d-f480b1885785 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.843250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1f59f-a623-4a45-868d-f480b1885785 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.849120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1f59f-a623-4a45-868d-f480b1885785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.849291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 248/489] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:01.849291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3b58-67d3-4e1f-9993-6a2d291714af None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.849291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3b58-67d3-4e1f-9993-6a2d291714af None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.849291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3b58-67d3-4e1f-9993-6a2d291714af None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.852928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 249/490] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:01.853517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb3b58-67d3-4e1f-9993-6a2d291714af None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:01.856293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1f59f-a623-4a45-868d-f480b1885785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:01.857886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1f59f-a623-4a45-868d-f480b1885785 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-19b1f59f-a623-4a45-868d-f480b1885785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:01.873979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:01.874944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:01.876745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:01.876745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:01.876745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:01.876745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 242/491] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:01.877902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118ffa3-95dd-4d36-b7dd-d0032479b13b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:01.877902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118ffa3-95dd-4d36-b7dd-d0032479b13b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:01.877902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118ffa3-95dd-4d36-b7dd-d0032479b13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:01.902985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f118ffa3-95dd-4d36-b7dd-d0032479b13b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:01.904139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f118ffa3-95dd-4d36-b7dd-d0032479b13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:01.946396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:01.980196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb3b58-67d3-4e1f-9993-6a2d291714af None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.000234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:02.014264 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 243/492] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:02.014264 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118ffa3-95dd-4d36-b7dd-d0032479b13b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.035451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733cf24-3933-42be-81af-301fc96496fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.036064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733cf24-3933-42be-81af-301fc96496fb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.036343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733cf24-3933-42be-81af-301fc96496fb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.036645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733cf24-3933-42be-81af-301fc96496fb None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.037339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733cf24-3933-42be-81af-301fc96496fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:02.037643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733cf24-3933-42be-81af-301fc96496fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:02.039066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733cf24-3933-42be-81af-301fc96496fb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:02.057556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 250/493] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:02.067842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733cf24-3933-42be-81af-301fc96496fb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.173408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118ffa3-95dd-4d36-b7dd-d0032479b13b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.205469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733cf24-3933-42be-81af-301fc96496fb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.215244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b55c95-94e7-4c55-a20f-de48e6dc2624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.215867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b55c95-94e7-4c55-a20f-de48e6dc2624 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.216145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b55c95-94e7-4c55-a20f-de48e6dc2624 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.216418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b55c95-94e7-4c55-a20f-de48e6dc2624 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.217076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b55c95-94e7-4c55-a20f-de48e6dc2624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:02.217381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b55c95-94e7-4c55-a20f-de48e6dc2624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:02.219695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b55c95-94e7-4c55-a20f-de48e6dc2624 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:02.238244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 244/494] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:02.239339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b55c95-94e7-4c55-a20f-de48e6dc2624 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.248375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b42d4-0362-428f-b2f4-2bf5634b4573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.249723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b42d4-0362-428f-b2f4-2bf5634b4573 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.250103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b42d4-0362-428f-b2f4-2bf5634b4573 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.250455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b42d4-0362-428f-b2f4-2bf5634b4573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.254083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:02.266489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b42d4-0362-428f-b2f4-2bf5634b4573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:02.268318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b42d4-0362-428f-b2f4-2bf5634b4573 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d44b42d4-0362-428f-b2f4-2bf5634b4573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:02.291613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:02.292577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:02.293420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:02.293420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:02.293420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:02.296453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 251/495] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:02.298324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b42d4-0362-428f-b2f4-2bf5634b4573 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.331807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:02.381727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b55c95-94e7-4c55-a20f-de48e6dc2624 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.420476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b42d4-0362-428f-b2f4-2bf5634b4573 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.423875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a198a-09cf-4a37-8edc-f33e68924ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.425077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a198a-09cf-4a37-8edc-f33e68924ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.425077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a198a-09cf-4a37-8edc-f33e68924ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.425077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a198a-09cf-4a37-8edc-f33e68924ad5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.440289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a198a-09cf-4a37-8edc-f33e68924ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-839a198a-09cf-4a37-8edc-f33e68924ad5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:02.445634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:02.446780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:02.447574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 245/496] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:02.448317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a198a-09cf-4a37-8edc-f33e68924ad5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.477568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3fd2d-e447-4c06-a968-64d19e9f339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.479030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3fd2d-e447-4c06-a968-64d19e9f339f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.479779 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3fd2d-e447-4c06-a968-64d19e9f339f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.480260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3fd2d-e447-4c06-a968-64d19e9f339f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.491534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3fd2d-e447-4c06-a968-64d19e9f339f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:02.493465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3fd2d-e447-4c06-a968-64d19e9f339f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-44d3fd2d-e447-4c06-a968-64d19e9f339f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:02.512093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:02.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:05:02.514003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:02.515323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 252/497] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:02.515428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3fd2d-e447-4c06-a968-64d19e9f339f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.570350 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a198a-09cf-4a37-8edc-f33e68924ad5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.573722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:02.595591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:02.612019 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bdf40-9fff-4219-a996-d1cf9a8430d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.612634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bdf40-9fff-4219-a996-d1cf9a8430d5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.612907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bdf40-9fff-4219-a996-d1cf9a8430d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.613239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bdf40-9fff-4219-a996-d1cf9a8430d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.613908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bdf40-9fff-4219-a996-d1cf9a8430d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:02.614211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bdf40-9fff-4219-a996-d1cf9a8430d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:02.615582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bdf40-9fff-4219-a996-d1cf9a8430d5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:02.627652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 246/498] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:02.628481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bdf40-9fff-4219-a996-d1cf9a8430d5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.637436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3fd2d-e447-4c06-a968-64d19e9f339f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.676135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6c0cb-d5e3-41ba-a853-49d9470cb6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.676680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6c0cb-d5e3-41ba-a853-49d9470cb6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.676960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6c0cb-d5e3-41ba-a853-49d9470cb6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.677241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6c0cb-d5e3-41ba-a853-49d9470cb6c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.690053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6c0cb-d5e3-41ba-a853-49d9470cb6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-67d6c0cb-d5e3-41ba-a853-49d9470cb6c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:02.695333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:02.696652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:02.697455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 253/499] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:02.698678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6c0cb-d5e3-41ba-a853-49d9470cb6c4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.750180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bdf40-9fff-4219-a996-d1cf9a8430d5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.791290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca672595-c976-482d-a27e-c8ca09a9719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.792205 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca672595-c976-482d-a27e-c8ca09a9719c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.792483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca672595-c976-482d-a27e-c8ca09a9719c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.792993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca672595-c976-482d-a27e-c8ca09a9719c None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.793438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca672595-c976-482d-a27e-c8ca09a9719c None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:02.793738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca672595-c976-482d-a27e-c8ca09a9719c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:02.795152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca672595-c976-482d-a27e-c8ca09a9719c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:02.804273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 247/500] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:02.805097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca672595-c976-482d-a27e-c8ca09a9719c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.814629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6c0cb-d5e3-41ba-a853-49d9470cb6c4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.822410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:02.853476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6274e80-7889-40ce-9dd1-a4978086fc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.854400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274e80-7889-40ce-9dd1-a4978086fc2d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.854764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274e80-7889-40ce-9dd1-a4978086fc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.855064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274e80-7889-40ce-9dd1-a4978086fc2d None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.855890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6274e80-7889-40ce-9dd1-a4978086fc2d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:02.856994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6274e80-7889-40ce-9dd1-a4978086fc2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:02.858552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6274e80-7889-40ce-9dd1-a4978086fc2d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:02.860841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:02.899864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 254/501] 10.222.0.83 () {56 vars in 1137 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:02.900900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6274e80-7889-40ce-9dd1-a4978086fc2d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:02.929437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca672595-c976-482d-a27e-c8ca09a9719c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:02.966045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19313b8f-7759-4e45-a3da-7e78ecec72cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:02.966729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19313b8f-7759-4e45-a3da-7e78ecec72cb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.966945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19313b8f-7759-4e45-a3da-7e78ecec72cb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.967369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19313b8f-7759-4e45-a3da-7e78ecec72cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.975888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19313b8f-7759-4e45-a3da-7e78ecec72cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:02.977319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19313b8f-7759-4e45-a3da-7e78ecec72cb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-19313b8f-7759-4e45-a3da-7e78ecec72cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:02.990153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:02.996777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 248/502] 10.222.0.83 () {56 vars in 1093 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eef07a-6ee0-4ce4-92b1-9e2bda3fe4e3 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:02.997922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eef07a-6ee0-4ce4-92b1-9e2bda3fe4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:02.998752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eef07a-6ee0-4ce4-92b1-9e2bda3fe4e3 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:02.998752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 249/503] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:02.998752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eef07a-6ee0-4ce4-92b1-9e2bda3fe4e3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:03.018196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6274e80-7889-40ce-9dd1-a4978086fc2d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:03.053863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd54649-f126-48b9-badf-67c8057f31f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:03.054490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd54649-f126-48b9-badf-67c8057f31f3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.054761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd54649-f126-48b9-badf-67c8057f31f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.055037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd54649-f126-48b9-badf-67c8057f31f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.058916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd54649-f126-48b9-badf-67c8057f31f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:03.058916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd54649-f126-48b9-badf-67c8057f31f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:03.058916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd54649-f126-48b9-badf-67c8057f31f3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:03.066680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 255/504] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:03.067498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd54649-f126-48b9-badf-67c8057f31f3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:03.096654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:03.108941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eef07a-6ee0-4ce4-92b1-9e2bda3fe4e3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:03.110740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:03.146996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce107e08-6663-4009-b552-cc6d5e0806e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:03.147635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107e08-6663-4009-b552-cc6d5e0806e1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.147917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107e08-6663-4009-b552-cc6d5e0806e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.148184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107e08-6663-4009-b552-cc6d5e0806e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.161429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce107e08-6663-4009-b552-cc6d5e0806e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:03.163894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce107e08-6663-4009-b552-cc6d5e0806e1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-ce107e08-6663-4009-b552-cc6d5e0806e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:03.184294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:03.185895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:03.186831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:03.186831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:03.186831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:03.186831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 250/505] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:03.186831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce107e08-6663-4009-b552-cc6d5e0806e1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:03.194961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd54649-f126-48b9-badf-67c8057f31f3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:03.235443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b315c52-948a-456d-9097-9b5189f76f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:03.236049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b315c52-948a-456d-9097-9b5189f76f05 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.236325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b315c52-948a-456d-9097-9b5189f76f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.236595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b315c52-948a-456d-9097-9b5189f76f05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.302751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce107e08-6663-4009-b552-cc6d5e0806e1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:03.308009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b315c52-948a-456d-9097-9b5189f76f05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:03.309487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b315c52-948a-456d-9097-9b5189f76f05 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:03.339557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2fbea-0eb8-4cf2-8cc4-0cc9b600835d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:03.340116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:03.340225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fbea-0eb8-4cf2-8cc4-0cc9b600835d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.340436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fbea-0eb8-4cf2-8cc4-0cc9b600835d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.340712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fbea-0eb8-4cf2-8cc4-0cc9b600835d None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.341452 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2fbea-0eb8-4cf2-8cc4-0cc9b600835d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:03.341758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2fbea-0eb8-4cf2-8cc4-0cc9b600835d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:03.343289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2fbea-0eb8-4cf2-8cc4-0cc9b600835d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:03.382031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 251/506] 10.222.0.83 () {56 vars in 1133 bytes} [Wed Aug 24 11:05:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:03.383363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b0b3c-6ac7-4d19-bf34-c60ab8c93b75 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.383661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b0b3c-6ac7-4d19-bf34-c60ab8c93b75 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.383934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b0b3c-6ac7-4d19-bf34-c60ab8c93b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.402712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-399b0b3c-6ac7-4d19-bf34-c60ab8c93b75 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:03.403931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-399b0b3c-6ac7-4d19-bf34-c60ab8c93b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:03.413841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:03.421329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 256/507] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:03.425103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b315c52-948a-456d-9097-9b5189f76f05 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:03.477633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 252/508] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:03.478443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b0b3c-6ac7-4d19-bf34-c60ab8c93b75 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:03.530347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b315c52-948a-456d-9097-9b5189f76f05 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:03.561698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:03.573102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede7242-1d40-47d0-a2bc-b41c5e67fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:03.573828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede7242-1d40-47d0-a2bc-b41c5e67fc45 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.574137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede7242-1d40-47d0-a2bc-b41c5e67fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.574403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede7242-1d40-47d0-a2bc-b41c5e67fc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.589223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede7242-1d40-47d0-a2bc-b41c5e67fc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:03.591549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede7242-1d40-47d0-a2bc-b41c5e67fc45 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:03.592958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b0b3c-6ac7-4d19-bf34-c60ab8c93b75 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-aede7242-1d40-47d0-a2bc-b41c5e67fc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:03.612932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:03.614266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:03.615375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:03.615375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:03.615375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:03.618026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 257/509] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:03.618997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede7242-1d40-47d0-a2bc-b41c5e67fc45 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:03.635746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21b814-9aae-41ba-ab5e-83893f950b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:03.636684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21b814-9aae-41ba-ab5e-83893f950b4d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.637032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21b814-9aae-41ba-ab5e-83893f950b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.637369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21b814-9aae-41ba-ab5e-83893f950b4d None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.638117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21b814-9aae-41ba-ab5e-83893f950b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:03.638485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21b814-9aae-41ba-ab5e-83893f950b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:03.640051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21b814-9aae-41ba-ab5e-83893f950b4d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:03.657717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 253/510] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:03.657717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21b814-9aae-41ba-ab5e-83893f950b4d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:03.661107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:03.731150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede7242-1d40-47d0-a2bc-b41c5e67fc45 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:03.766787 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848eeec-37e1-443b-9630-19e96902a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:03.767506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848eeec-37e1-443b-9630-19e96902a461 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.767791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848eeec-37e1-443b-9630-19e96902a461 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.768064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848eeec-37e1-443b-9630-19e96902a461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.769050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21b814-9aae-41ba-ab5e-83893f950b4d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:03.803029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac3c7a-d1eb-44ec-af25-b40c476a1d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:03.803733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac3c7a-d1eb-44ec-af25-b40c476a1d97 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.803918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac3c7a-d1eb-44ec-af25-b40c476a1d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.804189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac3c7a-d1eb-44ec-af25-b40c476a1d97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.815638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac3c7a-d1eb-44ec-af25-b40c476a1d97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:03.816709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:03.817081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac3c7a-d1eb-44ec-af25-b40c476a1d97 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-41ac3c7a-d1eb-44ec-af25-b40c476a1d97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:03.831425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:03.832352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:03.833266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:03.833266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:03.833266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:03.833266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 254/511] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:03.833560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac3c7a-d1eb-44ec-af25-b40c476a1d97 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:03.842037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848eeec-37e1-443b-9630-19e96902a461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:03.844051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848eeec-37e1-443b-9630-19e96902a461 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:03.890290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:03.938240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac3c7a-d1eb-44ec-af25-b40c476a1d97 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:03.948353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 258/512] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:03.949328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848eeec-37e1-443b-9630-19e96902a461 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:03.979566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149cca0-194e-42a9-8f1e-229fe0827b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:03.980177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149cca0-194e-42a9-8f1e-229fe0827b8f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:03.980460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149cca0-194e-42a9-8f1e-229fe0827b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:03.980732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149cca0-194e-42a9-8f1e-229fe0827b8f None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:03.981437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149cca0-194e-42a9-8f1e-229fe0827b8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:03.981681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149cca0-194e-42a9-8f1e-229fe0827b8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:03.983218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149cca0-194e-42a9-8f1e-229fe0827b8f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:03.998286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 255/513] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:03.999045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149cca0-194e-42a9-8f1e-229fe0827b8f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.033601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:04.056015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848eeec-37e1-443b-9630-19e96902a461 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.092331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dcea1-f1a3-4bee-a62c-d492966f508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.092971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dcea1-f1a3-4bee-a62c-d492966f508c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.093256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dcea1-f1a3-4bee-a62c-d492966f508c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.093532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dcea1-f1a3-4bee-a62c-d492966f508c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.101825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149cca0-194e-42a9-8f1e-229fe0827b8f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.102128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dcea1-f1a3-4bee-a62c-d492966f508c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.103673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dcea1-f1a3-4bee-a62c-d492966f508c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-465dcea1-f1a3-4bee-a62c-d492966f508c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:04.116060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:04.117072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 259/514] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:04.117990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dcea1-f1a3-4bee-a62c-d492966f508c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.120191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:04.139691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e00d13-0ae8-4543-ab98-401f7d866b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.140320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e00d13-0ae8-4543-ab98-401f7d866b29 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.140601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e00d13-0ae8-4543-ab98-401f7d866b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.140872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e00d13-0ae8-4543-ab98-401f7d866b29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.154420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e00d13-0ae8-4543-ab98-401f7d866b29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.156061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e00d13-0ae8-4543-ab98-401f7d866b29 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-75e00d13-0ae8-4543-ab98-401f7d866b29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:04.171544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:04.173108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:04.174095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:04.174095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:04.174095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:04.174095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 256/515] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:04.174309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e00d13-0ae8-4543-ab98-401f7d866b29 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.229292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dcea1-f1a3-4bee-a62c-d492966f508c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.273156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06699332-e78d-4bb8-a379-1d5697b1bc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.273729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06699332-e78d-4bb8-a379-1d5697b1bc5e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.274040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06699332-e78d-4bb8-a379-1d5697b1bc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.274319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06699332-e78d-4bb8-a379-1d5697b1bc5e None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.274963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06699332-e78d-4bb8-a379-1d5697b1bc5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:04.275263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06699332-e78d-4bb8-a379-1d5697b1bc5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.276640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06699332-e78d-4bb8-a379-1d5697b1bc5e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.286572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:04.289914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 260/516] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:04.290722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06699332-e78d-4bb8-a379-1d5697b1bc5e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.293514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e00d13-0ae8-4543-ab98-401f7d866b29 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.330648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0267fc5-f3ef-4923-ba0d-663a78a412b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.331323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0267fc5-f3ef-4923-ba0d-663a78a412b9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.331627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0267fc5-f3ef-4923-ba0d-663a78a412b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.331948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0267fc5-f3ef-4923-ba0d-663a78a412b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.343671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0267fc5-f3ef-4923-ba0d-663a78a412b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.345707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0267fc5-f3ef-4923-ba0d-663a78a412b9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a0267fc5-f3ef-4923-ba0d-663a78a412b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:04.360472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:04.361557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:04.362455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:04.362455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:04.362455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:04.362455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 257/517] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:04.363074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0267fc5-f3ef-4923-ba0d-663a78a412b9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.382862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:04.400202 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06699332-e78d-4bb8-a379-1d5697b1bc5e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.437988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73840-24e8-4eb7-b858-e269d493c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.439394 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73840-24e8-4eb7-b858-e269d493c4ec None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.439672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73840-24e8-4eb7-b858-e269d493c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.439946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73840-24e8-4eb7-b858-e269d493c4ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.448429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab73840-24e8-4eb7-b858-e269d493c4ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.450131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab73840-24e8-4eb7-b858-e269d493c4ec None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-eab73840-24e8-4eb7-b858-e269d493c4ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:04.462684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:04.463845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 261/518] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:04.464745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73840-24e8-4eb7-b858-e269d493c4ec None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.476088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0267fc5-f3ef-4923-ba0d-663a78a412b9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.512174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2e7ae-ce14-4a7e-820f-2fa4e75509a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.512717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2e7ae-ce14-4a7e-820f-2fa4e75509a2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.512977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2e7ae-ce14-4a7e-820f-2fa4e75509a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.513242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2e7ae-ce14-4a7e-820f-2fa4e75509a2 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.514004 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2e7ae-ce14-4a7e-820f-2fa4e75509a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:04.514302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2e7ae-ce14-4a7e-820f-2fa4e75509a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.515743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2e7ae-ce14-4a7e-820f-2fa4e75509a2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.526004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:04.535270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 258/519] 10.222.0.83 () {56 vars in 1137 bytes} [Wed Aug 24 11:05:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:04.536149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2e7ae-ce14-4a7e-820f-2fa4e75509a2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.573716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73840-24e8-4eb7-b858-e269d493c4ec None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.610086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee2af6-c594-4dcb-b30f-c7854c7bd7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.610697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee2af6-c594-4dcb-b30f-c7854c7bd7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.610978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee2af6-c594-4dcb-b30f-c7854c7bd7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.611256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee2af6-c594-4dcb-b30f-c7854c7bd7f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.611999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee2af6-c594-4dcb-b30f-c7854c7bd7f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:04.612254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee2af6-c594-4dcb-b30f-c7854c7bd7f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.613763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee2af6-c594-4dcb-b30f-c7854c7bd7f4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.617602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:04.626569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 262/520] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:04.635369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee2af6-c594-4dcb-b30f-c7854c7bd7f4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.642959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2e7ae-ce14-4a7e-820f-2fa4e75509a2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.678971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38709d6-d92f-48bf-b788-423f8d28dfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.679607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38709d6-d92f-48bf-b788-423f8d28dfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.679883 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38709d6-d92f-48bf-b788-423f8d28dfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.680265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38709d6-d92f-48bf-b788-423f8d28dfa8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.688532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38709d6-d92f-48bf-b788-423f8d28dfa8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.690037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38709d6-d92f-48bf-b788-423f8d28dfa8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a38709d6-d92f-48bf-b788-423f8d28dfa8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:04.702199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:04.703271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:04.704894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:04.704894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:04.704894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:04.704894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:04.704894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:04.704894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:04.704894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:04.704894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:04.704894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 259/521] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:04.705440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38709d6-d92f-48bf-b788-423f8d28dfa8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.736855 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee2af6-c594-4dcb-b30f-c7854c7bd7f4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.758459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:04.777420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9c9c8-c855-4ca9-99cc-e79f29f705bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.777974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9c9c8-c855-4ca9-99cc-e79f29f705bb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.778262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9c9c8-c855-4ca9-99cc-e79f29f705bb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.778532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9c9c8-c855-4ca9-99cc-e79f29f705bb None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.779215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9c9c8-c855-4ca9-99cc-e79f29f705bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:04.779514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9c9c8-c855-4ca9-99cc-e79f29f705bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.780927 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9c9c8-c855-4ca9-99cc-e79f29f705bb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.792272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 263/522] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:04.793174 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9c9c8-c855-4ca9-99cc-e79f29f705bb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.815044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38709d6-d92f-48bf-b788-423f8d28dfa8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.849624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6deaf-1345-4e08-888b-1eddc7fe9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.850258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6deaf-1345-4e08-888b-1eddc7fe9893 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.850542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6deaf-1345-4e08-888b-1eddc7fe9893 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.850819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6deaf-1345-4e08-888b-1eddc7fe9893 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.851553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6deaf-1345-4e08-888b-1eddc7fe9893 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:04.851864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6deaf-1345-4e08-888b-1eddc7fe9893 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:04.853323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6deaf-1345-4e08-888b-1eddc7fe9893 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:04.856056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:04.871514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 260/523] 10.222.0.83 () {56 vars in 1139 bytes} [Wed Aug 24 11:05:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:04.872633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6deaf-1345-4e08-888b-1eddc7fe9893 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:04.902010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9c9c8-c855-4ca9-99cc-e79f29f705bb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.938990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816f78e-9cf7-4bba-b217-ed058da39184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:04.939606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816f78e-9cf7-4bba-b217-ed058da39184 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:04.939908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816f78e-9cf7-4bba-b217-ed058da39184 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:04.940183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816f78e-9cf7-4bba-b217-ed058da39184 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:04.983397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6deaf-1345-4e08-888b-1eddc7fe9893 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:04.991038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:05.012969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816f78e-9cf7-4bba-b217-ed058da39184 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:05.014524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816f78e-9cf7-4bba-b217-ed058da39184 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:05.019239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34995156-6376-40ef-b2fc-2683d6e7bb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:05.019845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34995156-6376-40ef-b2fc-2683d6e7bb66 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.020122 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34995156-6376-40ef-b2fc-2683d6e7bb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.020404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34995156-6376-40ef-b2fc-2683d6e7bb66 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.021040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34995156-6376-40ef-b2fc-2683d6e7bb66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:05.022498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34995156-6376-40ef-b2fc-2683d6e7bb66 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:05.116874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 264/524] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:05.117750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816f78e-9cf7-4bba-b217-ed058da39184 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:05.164149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 261/525] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:05:04 2022] POST /identity/v3/users => generated 247 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:05.165216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34995156-6376-40ef-b2fc-2683d6e7bb66 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:05.217223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:05.243566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:05.257668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816f78e-9cf7-4bba-b217-ed058da39184 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:05.301677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97cf39-437a-4bd1-8ea1-6808c658cf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:05.302316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97cf39-437a-4bd1-8ea1-6808c658cf69 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.302596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97cf39-437a-4bd1-8ea1-6808c658cf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.302947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97cf39-437a-4bd1-8ea1-6808c658cf69 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.303576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97cf39-437a-4bd1-8ea1-6808c658cf69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:05.303880 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97cf39-437a-4bd1-8ea1-6808c658cf69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:05.305336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97cf39-437a-4bd1-8ea1-6808c658cf69 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:05.306978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34995156-6376-40ef-b2fc-2683d6e7bb66 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:05.327729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 265/526] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:05.328246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97cf39-437a-4bd1-8ea1-6808c658cf69 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:05.349832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acbe73-435d-48af-a0ef-f027986c162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:05.350499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acbe73-435d-48af-a0ef-f027986c162f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.350775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acbe73-435d-48af-a0ef-f027986c162f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.351047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acbe73-435d-48af-a0ef-f027986c162f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.359022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acbe73-435d-48af-a0ef-f027986c162f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:05.361973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acbe73-435d-48af-a0ef-f027986c162f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-15acbe73-435d-48af-a0ef-f027986c162f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:05.377830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:05.378727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:05.379555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:05.379555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:05.379555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:05.379555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:05.379555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:05.379555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:05.379555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:05.379555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:05.379555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 262/527] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:05.380004 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acbe73-435d-48af-a0ef-f027986c162f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:05.451286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97cf39-437a-4bd1-8ea1-6808c658cf69 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:05.493002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1297e-b77e-40d1-a882-8d7f226fda12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:05.493770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1297e-b77e-40d1-a882-8d7f226fda12 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.494018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1297e-b77e-40d1-a882-8d7f226fda12 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.494290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1297e-b77e-40d1-a882-8d7f226fda12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.499387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acbe73-435d-48af-a0ef-f027986c162f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:05.504005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:05.510447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:05.535150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51da6d-6c34-416f-b1bd-8fa5a561d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:05.535806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51da6d-6c34-416f-b1bd-8fa5a561d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.536071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51da6d-6c34-416f-b1bd-8fa5a561d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.536343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51da6d-6c34-416f-b1bd-8fa5a561d2f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.547374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51da6d-6c34-416f-b1bd-8fa5a561d2f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:05.549343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51da6d-6c34-416f-b1bd-8fa5a561d2f0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8c51da6d-6c34-416f-b1bd-8fa5a561d2f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:05.564388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:05.565367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:05.566260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:05.566260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:05.566260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:05.566260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 263/528] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:05.566472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51da6d-6c34-416f-b1bd-8fa5a561d2f0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:05.586773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1297e-b77e-40d1-a882-8d7f226fda12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:05.588446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1297e-b77e-40d1-a882-8d7f226fda12 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:05.691683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51da6d-6c34-416f-b1bd-8fa5a561d2f0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:05.705218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 266/529] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:05.705928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1297e-b77e-40d1-a882-8d7f226fda12 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:05.730888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542863f-f70f-4058-b8f7-bc2f2a7937a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:05.732025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542863f-f70f-4058-b8f7-bc2f2a7937a6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.732552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542863f-f70f-4058-b8f7-bc2f2a7937a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.732994 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542863f-f70f-4058-b8f7-bc2f2a7937a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.734016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542863f-f70f-4058-b8f7-bc2f2a7937a6 None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=f1f214c2b77d4cbab2f15122ba21bf38) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:05.734603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542863f-f70f-4058-b8f7-bc2f2a7937a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:05.736580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542863f-f70f-4058-b8f7-bc2f2a7937a6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:05.742983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 264/530] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:05 2022] GET /identity/v3/role_assignments?user.id=f1f214c2b77d4cbab2f15122ba21bf38&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:05.743631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542863f-f70f-4058-b8f7-bc2f2a7937a6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:05.748445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:05.783303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:05.850867 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1297e-b77e-40d1-a882-8d7f226fda12 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:05.893280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f8507-153e-4bd4-9267-edf895d2750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:05.894847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f8507-153e-4bd4-9267-edf895d2750c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.895146 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f8507-153e-4bd4-9267-edf895d2750c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.895388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542863f-f70f-4058-b8f7-bc2f2a7937a6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:05.895512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f8507-153e-4bd4-9267-edf895d2750c None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.896190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f8507-153e-4bd4-9267-edf895d2750c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:05.896505 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f8507-153e-4bd4-9267-edf895d2750c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:05.898842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f8507-153e-4bd4-9267-edf895d2750c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:05.914104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 267/531] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:05.915379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f8507-153e-4bd4-9267-edf895d2750c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:05.939216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cba50-9e86-4d8a-99fc-fe9249cc7415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:05.939823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cba50-9e86-4d8a-99fc-fe9249cc7415 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:05.940098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cba50-9e86-4d8a-99fc-fe9249cc7415 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:05.940365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cba50-9e86-4d8a-99fc-fe9249cc7415 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:05.941019 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cba50-9e86-4d8a-99fc-fe9249cc7415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:05.941319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cba50-9e86-4d8a-99fc-fe9249cc7415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:05.943401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cba50-9e86-4d8a-99fc-fe9249cc7415 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:05.963763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 265/532] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:05.965047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cba50-9e86-4d8a-99fc-fe9249cc7415 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:06.037075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:06.041481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f8507-153e-4bd4-9267-edf895d2750c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:06.079488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9407f-5372-4467-8c08-f5341a17ddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:06.080542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9407f-5372-4467-8c08-f5341a17ddd2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:06.081028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9407f-5372-4467-8c08-f5341a17ddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:06.081504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9407f-5372-4467-8c08-f5341a17ddd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:06.082291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:06.094619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9407f-5372-4467-8c08-f5341a17ddd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:06.095677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cba50-9e86-4d8a-99fc-fe9249cc7415 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:06.096641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9407f-5372-4467-8c08-f5341a17ddd2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-bec9407f-5372-4467-8c08-f5341a17ddd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:06.112828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:06.113782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:06.114984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:06.114984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:06.114984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:06.116648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 268/533] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:06.117668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9407f-5372-4467-8c08-f5341a17ddd2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:06.138695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9a50b-098c-4775-ad77-1ca9cf5cc202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:06.139309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9a50b-098c-4775-ad77-1ca9cf5cc202 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:06.139583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9a50b-098c-4775-ad77-1ca9cf5cc202 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:06.139884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9a50b-098c-4775-ad77-1ca9cf5cc202 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:06.140735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9a50b-098c-4775-ad77-1ca9cf5cc202 None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=32a28f9fb3134f33819e53e0c72c63b4) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:06.141096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9a50b-098c-4775-ad77-1ca9cf5cc202 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:06.143416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9a50b-098c-4775-ad77-1ca9cf5cc202 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:06.151411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 266/534] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:05 2022] GET /identity/v3/role_assignments?user.id=32a28f9fb3134f33819e53e0c72c63b4&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:06.152378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9a50b-098c-4775-ad77-1ca9cf5cc202 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:06.309059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9407f-5372-4467-8c08-f5341a17ddd2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:06.323990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9a50b-098c-4775-ad77-1ca9cf5cc202 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:06.353902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279968d9-7da4-4dd1-9c87-44da7e747d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:06.354722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279968d9-7da4-4dd1-9c87-44da7e747d59 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:06.355012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279968d9-7da4-4dd1-9c87-44da7e747d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:06.355283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279968d9-7da4-4dd1-9c87-44da7e747d59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:06.364584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279968d9-7da4-4dd1-9c87-44da7e747d59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:06.366373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279968d9-7da4-4dd1-9c87-44da7e747d59 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-279968d9-7da4-4dd1-9c87-44da7e747d59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:06.385142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:06.386301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:06.387379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:06.387379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:06.387379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:06.387379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:06.387379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:06.387379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:06.387379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:06.387379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:06.387379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 269/535] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:06.394958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279968d9-7da4-4dd1-9c87-44da7e747d59 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:06.415628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b033060-db51-4018-8614-eed12c1ec55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:06.416278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b033060-db51-4018-8614-eed12c1ec55d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:06.416556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b033060-db51-4018-8614-eed12c1ec55d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:06.416924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b033060-db51-4018-8614-eed12c1ec55d None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:06.417584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b033060-db51-4018-8614-eed12c1ec55d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:06.417907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b033060-db51-4018-8614-eed12c1ec55d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:06.419316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b033060-db51-4018-8614-eed12c1ec55d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:06.426483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:06.442178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 267/536] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:06.449899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b033060-db51-4018-8614-eed12c1ec55d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:06.473668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:06.626508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279968d9-7da4-4dd1-9c87-44da7e747d59 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:06.657590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b033060-db51-4018-8614-eed12c1ec55d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:06.682843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fb565-be41-420e-aa84-60b3b700fb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:06.683522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fb565-be41-420e-aa84-60b3b700fb42 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:06.683836 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fb565-be41-420e-aa84-60b3b700fb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:06.684179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fb565-be41-420e-aa84-60b3b700fb42 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:06.684997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fb565-be41-420e-aa84-60b3b700fb42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:06.685309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fb565-be41-420e-aa84-60b3b700fb42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:06.686809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fb565-be41-420e-aa84-60b3b700fb42 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:06.696905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ab833-a7d7-471a-b686-983ff325676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:06.697537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ab833-a7d7-471a-b686-983ff325676b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:06.697813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ab833-a7d7-471a-b686-983ff325676b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:06.698147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ab833-a7d7-471a-b686-983ff325676b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:06.706015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ab833-a7d7-471a-b686-983ff325676b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:06.707601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ab833-a7d7-471a-b686-983ff325676b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:06.712102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 270/537] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:06.719486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fb565-be41-420e-aa84-60b3b700fb42 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-fb9ab833-a7d7-471a-b686-983ff325676b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:06.720209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:06.721026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:06.721859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:06.721859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:06.721859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:06.721859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:06.721859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:06.721859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:06.721859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:06.721859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:06.721859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 268/538] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:06.730160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ab833-a7d7-471a-b686-983ff325676b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:06.838278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:06.855134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ab833-a7d7-471a-b686-983ff325676b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:06.894320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:06.935263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e51b21-e652-4137-a3bc-0566a37080d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:06.935890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e51b21-e652-4137-a3bc-0566a37080d7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:06.936165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e51b21-e652-4137-a3bc-0566a37080d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:06.936428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e51b21-e652-4137-a3bc-0566a37080d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:06.937128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e51b21-e652-4137-a3bc-0566a37080d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:06.937430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e51b21-e652-4137-a3bc-0566a37080d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:06.938913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e51b21-e652-4137-a3bc-0566a37080d7 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:06.952805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fb565-be41-420e-aa84-60b3b700fb42 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:06.974660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 269/539] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:06.980351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e51b21-e652-4137-a3bc-0566a37080d7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:06.993143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9de5d1-5487-439e-92fa-0d37ec8a636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:06.993757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9de5d1-5487-439e-92fa-0d37ec8a636c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:06.994067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9de5d1-5487-439e-92fa-0d37ec8a636c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:06.994362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9de5d1-5487-439e-92fa-0d37ec8a636c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:06.995241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9de5d1-5487-439e-92fa-0d37ec8a636c None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=f9dd6f187dd84f04866da64732b242d7) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:06.995552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9de5d1-5487-439e-92fa-0d37ec8a636c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:06.997156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9de5d1-5487-439e-92fa-0d37ec8a636c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:07.005181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 271/540] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:06 2022] GET /identity/v3/role_assignments?user.id=f9dd6f187dd84f04866da64732b242d7&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:07.117298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e51b21-e652-4137-a3bc-0566a37080d7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:07.152055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:07.164266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b9c42-0265-4446-b4a1-1ceba1e6502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:07.164893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b9c42-0265-4446-b4a1-1ceba1e6502b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:07.165170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b9c42-0265-4446-b4a1-1ceba1e6502b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:07.165440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b9c42-0265-4446-b4a1-1ceba1e6502b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:07.170742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b9c42-0265-4446-b4a1-1ceba1e6502b None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=b0145242ef784168bada8348ccd95f22) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:07.170742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b9c42-0265-4446-b4a1-1ceba1e6502b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:07.170742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b9c42-0265-4446-b4a1-1ceba1e6502b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:07.178843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 270/541] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:06 2022] GET /identity/v3/role_assignments?user.id=b0145242ef784168bada8348ccd95f22&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:07.354981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3261cd-1614-4747-aeb1-9b13d50a085c None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:07.355323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3261cd-1614-4747-aeb1-9b13d50a085c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:07.355526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3261cd-1614-4747-aeb1-9b13d50a085c None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:07.356883 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 272/542] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:07.364646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdbf80-c756-4537-8e85-b58fddc8d68d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:07.365163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdbf80-c756-4537-8e85-b58fddc8d68d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:07.365163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdbf80-c756-4537-8e85-b58fddc8d68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:07.387215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1ffdbf80-c756-4537-8e85-b58fddc8d68d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:07.388423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1ffdbf80-c756-4537-8e85-b58fddc8d68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:07.477826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 271/543] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:07.871931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3261cd-1614-4747-aeb1-9b13d50a085c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:08.001768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3261cd-1614-4747-aeb1-9b13d50a085c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:08.041830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20285f-760e-4a4b-89ed-8b57f334d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:08.042517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20285f-760e-4a4b-89ed-8b57f334d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.042797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20285f-760e-4a4b-89ed-8b57f334d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.043069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20285f-760e-4a4b-89ed-8b57f334d2b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.046169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:08.061251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20285f-760e-4a4b-89ed-8b57f334d2b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:08.064204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66744714-c96c-49ba-a535-ba784b8da739 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.064490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66744714-c96c-49ba-a535-ba784b8da739 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.064769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66744714-c96c-49ba-a535-ba784b8da739 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7a20285f-760e-4a4b-89ed-8b57f334d2b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:08.066665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:08.067875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:08.068999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:08.069750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 272/544] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:08.069830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 273/545] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:08.075846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66744714-c96c-49ba-a535-ba784b8da739 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:08.081156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278560b-4532-44ec-863c-3b0e6a920e3a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.081459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278560b-4532-44ec-863c-3b0e6a920e3a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.081775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278560b-4532-44ec-863c-3b0e6a920e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.118651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9278560b-4532-44ec-863c-3b0e6a920e3a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:08.119894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9278560b-4532-44ec-863c-3b0e6a920e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:08.203236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66744714-c96c-49ba-a535-ba784b8da739 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:08.232559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:08.245716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160798cd-16c0-40c0-b117-0d24af676bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:08.247220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160798cd-16c0-40c0-b117-0d24af676bfb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.247718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160798cd-16c0-40c0-b117-0d24af676bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.248565 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160798cd-16c0-40c0-b117-0d24af676bfb None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.249785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160798cd-16c0-40c0-b117-0d24af676bfb None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:08.251173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160798cd-16c0-40c0-b117-0d24af676bfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:08.252909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160798cd-16c0-40c0-b117-0d24af676bfb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:08.263398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 273/546] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:08.270155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160798cd-16c0-40c0-b117-0d24af676bfb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:08.271451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 274/547] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:05:08.395069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160798cd-16c0-40c0-b117-0d24af676bfb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:08.441741 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff93819-3361-4c11-aebf-0defa5c9145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:08.442435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff93819-3361-4c11-aebf-0defa5c9145e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.442753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff93819-3361-4c11-aebf-0defa5c9145e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.443064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff93819-3361-4c11-aebf-0defa5c9145e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.456330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff93819-3361-4c11-aebf-0defa5c9145e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:08.457897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff93819-3361-4c11-aebf-0defa5c9145e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-6ff93819-3361-4c11-aebf-0defa5c9145e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:08.486364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:08.487444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:08.488325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:08.488325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:08.488325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:08.488325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 274/548] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:08.494499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278560b-4532-44ec-863c-3b0e6a920e3a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:08.631631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278560b-4532-44ec-863c-3b0e6a920e3a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:08.659414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:08.671278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d0fb9-9513-46a0-915a-6b8a4088649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:08.680726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d0fb9-9513-46a0-915a-6b8a4088649f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.680726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d0fb9-9513-46a0-915a-6b8a4088649f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.680726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d0fb9-9513-46a0-915a-6b8a4088649f None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.681036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff93819-3361-4c11-aebf-0defa5c9145e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:08.681111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d0fb9-9513-46a0-915a-6b8a4088649f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:08.681690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d0fb9-9513-46a0-915a-6b8a4088649f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:08.683638 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d0fb9-9513-46a0-915a-6b8a4088649f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:08.704274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 275/549] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:08.706766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8d413-a014-4f37-af7c-3c4c3cf886b7 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.707199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8d413-a014-4f37-af7c-3c4c3cf886b7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.707682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8d413-a014-4f37-af7c-3c4c3cf886b7 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.710107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 276/550] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:08.712608 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8d413-a014-4f37-af7c-3c4c3cf886b7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:08.734572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:08.818007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff93819-3361-4c11-aebf-0defa5c9145e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:08.845932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8d413-a014-4f37-af7c-3c4c3cf886b7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:08.866160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ddb04-ea63-48ac-803f-50834f423809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:08.867927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ddb04-ea63-48ac-803f-50834f423809 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.868396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ddb04-ea63-48ac-803f-50834f423809 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.868836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ddb04-ea63-48ac-803f-50834f423809 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.885542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ddb04-ea63-48ac-803f-50834f423809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:08.888136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8d2a9-44cc-4fe5-9ae2-85d7d77a4b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:08.888894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8d2a9-44cc-4fe5-9ae2-85d7d77a4b88 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.889265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8d2a9-44cc-4fe5-9ae2-85d7d77a4b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.889612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8d2a9-44cc-4fe5-9ae2-85d7d77a4b88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e94ddb04-ea63-48ac-803f-50834f423809 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:08.893305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:08.894246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:08.895042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:08.898082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 275/551] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:08.899473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104111-d6cc-4d27-bd52-1c147c9229ef None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:08.899922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104111-d6cc-4d27-bd52-1c147c9229ef None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:08.900354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104111-d6cc-4d27-bd52-1c147c9229ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:08.904072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8d2a9-44cc-4fe5-9ae2-85d7d77a4b88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:08.905718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8d2a9-44cc-4fe5-9ae2-85d7d77a4b88 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-1aa8d2a9-44cc-4fe5-9ae2-85d7d77a4b88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:08.919763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:08.920763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:08.921678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 277/552] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:08.922863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8d2a9-44cc-4fe5-9ae2-85d7d77a4b88 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:08.924458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cb104111-d6cc-4d27-bd52-1c147c9229ef None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:08.925833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cb104111-d6cc-4d27-bd52-1c147c9229ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:09.001411 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:09.057706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 276/553] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:09.060393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb104111-d6cc-4d27-bd52-1c147c9229ef None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:09.086821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8d2a9-44cc-4fe5-9ae2-85d7d77a4b88 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:09.128464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:09.136365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508fe56-11a8-4217-bf6e-74acdf852643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:09.137002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508fe56-11a8-4217-bf6e-74acdf852643 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.137347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508fe56-11a8-4217-bf6e-74acdf852643 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.137689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508fe56-11a8-4217-bf6e-74acdf852643 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.138586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508fe56-11a8-4217-bf6e-74acdf852643 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:09.138766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508fe56-11a8-4217-bf6e-74acdf852643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:09.140501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508fe56-11a8-4217-bf6e-74acdf852643 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:09.163657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 278/554] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:09.170129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508fe56-11a8-4217-bf6e-74acdf852643 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:09.221304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb104111-d6cc-4d27-bd52-1c147c9229ef None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:09.268221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06637b4c-69b8-4bb9-916f-bf1b0aa16713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:09.269737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06637b4c-69b8-4bb9-916f-bf1b0aa16713 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.271316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06637b4c-69b8-4bb9-916f-bf1b0aa16713 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.272100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06637b4c-69b8-4bb9-916f-bf1b0aa16713 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.273204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06637b4c-69b8-4bb9-916f-bf1b0aa16713 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:09.274975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06637b4c-69b8-4bb9-916f-bf1b0aa16713 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:09.276767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06637b4c-69b8-4bb9-916f-bf1b0aa16713 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:09.301005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 277/555] 10.222.0.83 () {56 vars in 1135 bytes} [Wed Aug 24 11:05:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:09.310543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06637b4c-69b8-4bb9-916f-bf1b0aa16713 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:09.400637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508fe56-11a8-4217-bf6e-74acdf852643 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:09.409767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:09.446075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dad03b-7adc-4754-aa51-bfa119dfbba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:09.447014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dad03b-7adc-4754-aa51-bfa119dfbba1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.447345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dad03b-7adc-4754-aa51-bfa119dfbba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.447794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dad03b-7adc-4754-aa51-bfa119dfbba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.464059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dad03b-7adc-4754-aa51-bfa119dfbba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:09.466038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dad03b-7adc-4754-aa51-bfa119dfbba1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:09.473526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-19dad03b-7adc-4754-aa51-bfa119dfbba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:09.485910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:09.488454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:09.489791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:09.489791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:09.489791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:09.489791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 279/556] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:09.489791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dad03b-7adc-4754-aa51-bfa119dfbba1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:09.500310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06637b4c-69b8-4bb9-916f-bf1b0aa16713 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:09.547545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf9c67-dcd5-42f6-bbd1-f6e06291b083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:09.549102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf9c67-dcd5-42f6-bbd1-f6e06291b083 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.549386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf9c67-dcd5-42f6-bbd1-f6e06291b083 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.549662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf9c67-dcd5-42f6-bbd1-f6e06291b083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.629204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dad03b-7adc-4754-aa51-bfa119dfbba1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:09.643801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf9c67-dcd5-42f6-bbd1-f6e06291b083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:09.645272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf9c67-dcd5-42f6-bbd1-f6e06291b083 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:09.673703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2d5a8-4913-41ca-8590-a226b055f0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:09.674578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2d5a8-4913-41ca-8590-a226b055f0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.674865 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2d5a8-4913-41ca-8590-a226b055f0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.675170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2d5a8-4913-41ca-8590-a226b055f0b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.687518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2d5a8-4913-41ca-8590-a226b055f0b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-30c2d5a8-4913-41ca-8590-a226b055f0b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:09.692980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:09.694810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 280/557] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:09.695762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2d5a8-4913-41ca-8590-a226b055f0b7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:09.716153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:09.745142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:09.763799 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 278/558] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:05:09.769680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457d6eb-bcd5-4938-8ecc-34a853f05cae None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.769680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457d6eb-bcd5-4938-8ecc-34a853f05cae None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.769680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457d6eb-bcd5-4938-8ecc-34a853f05cae None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.769680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 279/559] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:09.769680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e9eb-cbaa-4528-bcfc-b20829bb03dc None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.769680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e9eb-cbaa-4528-bcfc-b20829bb03dc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.770246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e9eb-cbaa-4528-bcfc-b20829bb03dc None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.770541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 280/560] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:09.771276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e9eb-cbaa-4528-bcfc-b20829bb03dc None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:09.810960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2d5a8-4913-41ca-8590-a226b055f0b7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:09.851571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c8559-778e-47b9-a1d0-8686e4dcc663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:09.852184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c8559-778e-47b9-a1d0-8686e4dcc663 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.852465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c8559-778e-47b9-a1d0-8686e4dcc663 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.852778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c8559-778e-47b9-a1d0-8686e4dcc663 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.853465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c8559-778e-47b9-a1d0-8686e4dcc663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:09.853798 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c8559-778e-47b9-a1d0-8686e4dcc663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:09.855192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c8559-778e-47b9-a1d0-8686e4dcc663 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:09.869253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 281/561] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:09.870225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c8559-778e-47b9-a1d0-8686e4dcc663 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:09.888138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e9eb-cbaa-4528-bcfc-b20829bb03dc None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:09.923944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd9310-33b9-4950-bd41-86e9c224e0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:09.924555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd9310-33b9-4950-bd41-86e9c224e0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.924851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd9310-33b9-4950-bd41-86e9c224e0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.925181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd9310-33b9-4950-bd41-86e9c224e0b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.925779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd9310-33b9-4950-bd41-86e9c224e0b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:09.926112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd9310-33b9-4950-bd41-86e9c224e0b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:09.927499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd9310-33b9-4950-bd41-86e9c224e0b5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:09.936152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 281/562] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:09.937463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50e4e0-0737-4f54-a9f8-8586da25b233 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:09.937752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50e4e0-0737-4f54-a9f8-8586da25b233 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:09.938580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50e4e0-0737-4f54-a9f8-8586da25b233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:09.957074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2d50e4e0-0737-4f54-a9f8-8586da25b233 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:09.958347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2d50e4e0-0737-4f54-a9f8-8586da25b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:09.962678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:09.985037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c8559-778e-47b9-a1d0-8686e4dcc663 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.004840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:10.022568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1e1ca-a731-4c65-a9a2-8d12dc589850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.023216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1e1ca-a731-4c65-a9a2-8d12dc589850 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.023498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1e1ca-a731-4c65-a9a2-8d12dc589850 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.023773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1e1ca-a731-4c65-a9a2-8d12dc589850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.029064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 282/563] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:10.030550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89459e38-c06d-4ace-aba7-9f0b3ede402f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.030868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89459e38-c06d-4ace-aba7-9f0b3ede402f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.031373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89459e38-c06d-4ace-aba7-9f0b3ede402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.036211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1e1ca-a731-4c65-a9a2-8d12dc589850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:10.036729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1e1ca-a731-4c65-a9a2-8d12dc589850 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ebc1e1ca-a731-4c65-a9a2-8d12dc589850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:10.060438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:10.067913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:10.068799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:10.068799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:10.068799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:10.069069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-89459e38-c06d-4ace-aba7-9f0b3ede402f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:10.069145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 282/564] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:10.069145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1e1ca-a731-4c65-a9a2-8d12dc589850 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.069271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-89459e38-c06d-4ace-aba7-9f0b3ede402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:10.150149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 283/565] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:10.150149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89459e38-c06d-4ace-aba7-9f0b3ede402f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.199500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1e1ca-a731-4c65-a9a2-8d12dc589850 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.231051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:10.238344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7a667-47b1-4c8a-bb69-9c6def0cb280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.239032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7a667-47b1-4c8a-bb69-9c6def0cb280 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.239316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7a667-47b1-4c8a-bb69-9c6def0cb280 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.239586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7a667-47b1-4c8a-bb69-9c6def0cb280 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.249635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7a667-47b1-4c8a-bb69-9c6def0cb280 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:10.251863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7a667-47b1-4c8a-bb69-9c6def0cb280 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f2d7a667-47b1-4c8a-bb69-9c6def0cb280 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:10.267359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:10.268402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 283/566] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:10.270125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7a667-47b1-4c8a-bb69-9c6def0cb280 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.296411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89459e38-c06d-4ace-aba7-9f0b3ede402f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.309531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:10.340264 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e0b6b-e01a-402c-a85b-bb010cd2330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.340909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e0b6b-e01a-402c-a85b-bb010cd2330d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.341192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e0b6b-e01a-402c-a85b-bb010cd2330d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.341505 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e0b6b-e01a-402c-a85b-bb010cd2330d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.352827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e0b6b-e01a-402c-a85b-bb010cd2330d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:10.354363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e0b6b-e01a-402c-a85b-bb010cd2330d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-ae4e0b6b-e01a-402c-a85b-bb010cd2330d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:10.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:10.372128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:10.373622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:10.373622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:10.373622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:10.373622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 284/567] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:10.373622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e0b6b-e01a-402c-a85b-bb010cd2330d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.401599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7a667-47b1-4c8a-bb69-9c6def0cb280 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.443151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4cc34-870c-467c-b97b-6f785c643df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.444149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4cc34-870c-467c-b97b-6f785c643df7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.444149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4cc34-870c-467c-b97b-6f785c643df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.444351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4cc34-870c-467c-b97b-6f785c643df7 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.444949 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4cc34-870c-467c-b97b-6f785c643df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:10.445339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4cc34-870c-467c-b97b-6f785c643df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:10.446802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4cc34-870c-467c-b97b-6f785c643df7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:10.459837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 284/568] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:10.460521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4cc34-870c-467c-b97b-6f785c643df7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.495273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e0b6b-e01a-402c-a85b-bb010cd2330d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.513152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:10.530138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315746c-7a6a-4c4b-a91c-23509cdbb43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.531708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315746c-7a6a-4c4b-a91c-23509cdbb43c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.531708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315746c-7a6a-4c4b-a91c-23509cdbb43c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.531708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315746c-7a6a-4c4b-a91c-23509cdbb43c None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.532209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315746c-7a6a-4c4b-a91c-23509cdbb43c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:10.532512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315746c-7a6a-4c4b-a91c-23509cdbb43c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:10.533987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315746c-7a6a-4c4b-a91c-23509cdbb43c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:10.553279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 285/569] 10.222.0.83 () {56 vars in 1137 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:10.553754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315746c-7a6a-4c4b-a91c-23509cdbb43c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.564593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4cc34-870c-467c-b97b-6f785c643df7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.573697 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:10.599602 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe7e19-688a-4207-af06-09be477c6bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.600137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe7e19-688a-4207-af06-09be477c6bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.600425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe7e19-688a-4207-af06-09be477c6bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.600702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe7e19-688a-4207-af06-09be477c6bd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.611966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe7e19-688a-4207-af06-09be477c6bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-72fe7e19-688a-4207-af06-09be477c6bd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:10.616829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:10.617823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:10.618731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 285/570] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:10.619464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe7e19-688a-4207-af06-09be477c6bd0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.661860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315746c-7a6a-4c4b-a91c-23509cdbb43c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.696582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6d6b-11e7-42b4-b1da-c4153bd29273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.710186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6d6b-11e7-42b4-b1da-c4153bd29273 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.710186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6d6b-11e7-42b4-b1da-c4153bd29273 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.710186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6d6b-11e7-42b4-b1da-c4153bd29273 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.710186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c6d6b-11e7-42b4-b1da-c4153bd29273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:10.710186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c6d6b-11e7-42b4-b1da-c4153bd29273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:10.710186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c6d6b-11e7-42b4-b1da-c4153bd29273 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:10.712387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 286/571] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:10.713228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6d6b-11e7-42b4-b1da-c4153bd29273 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.727392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe7e19-688a-4207-af06-09be477c6bd0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.738263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:10.762133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086fed3-fadf-455f-b9ab-18a65b007c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.762758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086fed3-fadf-455f-b9ab-18a65b007c98 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.763035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086fed3-fadf-455f-b9ab-18a65b007c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.763306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086fed3-fadf-455f-b9ab-18a65b007c98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.774606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086fed3-fadf-455f-b9ab-18a65b007c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-8086fed3-fadf-455f-b9ab-18a65b007c98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:10.779343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:10.780321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:10.781165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 286/572] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:10.782552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086fed3-fadf-455f-b9ab-18a65b007c98 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.800945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:10.823891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6d6b-11e7-42b4-b1da-c4153bd29273 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.862618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf73a-953e-40a2-8d8f-f0b2d8a0939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.863170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf73a-953e-40a2-8d8f-f0b2d8a0939c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.863454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf73a-953e-40a2-8d8f-f0b2d8a0939c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.863729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf73a-953e-40a2-8d8f-f0b2d8a0939c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.870859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbf73a-953e-40a2-8d8f-f0b2d8a0939c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:10.872344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbf73a-953e-40a2-8d8f-f0b2d8a0939c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-90cbf73a-953e-40a2-8d8f-f0b2d8a0939c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:10.884650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:10.885957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 287/573] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:10.887142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf73a-953e-40a2-8d8f-f0b2d8a0939c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:10.896320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086fed3-fadf-455f-b9ab-18a65b007c98 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:10.938790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9ba2-fce4-4c38-8931-7ee3fac2d2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:10.939484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9ba2-fce4-4c38-8931-7ee3fac2d2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:10.939759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9ba2-fce4-4c38-8931-7ee3fac2d2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:10.940374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9ba2-fce4-4c38-8931-7ee3fac2d2b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:10.953271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d9ba2-fce4-4c38-8931-7ee3fac2d2b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:10.954687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d9ba2-fce4-4c38-8931-7ee3fac2d2b3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-690d9ba2-fce4-4c38-8931-7ee3fac2d2b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:10.982310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:10.983579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:10.985208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:10.985208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:10.985208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:10.985208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 287/574] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:10.985853 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9ba2-fce4-4c38-8931-7ee3fac2d2b3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.014212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:11.017204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf73a-953e-40a2-8d8f-f0b2d8a0939c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.056921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8aa6f-8f0a-42dc-8473-ed81cab8cfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.061111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8aa6f-8f0a-42dc-8473-ed81cab8cfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.061111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8aa6f-8f0a-42dc-8473-ed81cab8cfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.061111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8aa6f-8f0a-42dc-8473-ed81cab8cfa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.061111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8aa6f-8f0a-42dc-8473-ed81cab8cfa8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:11.061111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8aa6f-8f0a-42dc-8473-ed81cab8cfa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.061111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8aa6f-8f0a-42dc-8473-ed81cab8cfa8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.067447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:11.071528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 288/575] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:11.072460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8aa6f-8f0a-42dc-8473-ed81cab8cfa8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.115330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9ba2-fce4-4c38-8931-7ee3fac2d2b3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.151529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec029d-f600-4089-b99f-cef0b751f464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.152164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec029d-f600-4089-b99f-cef0b751f464 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.152442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec029d-f600-4089-b99f-cef0b751f464 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.152718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec029d-f600-4089-b99f-cef0b751f464 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.160826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec029d-f600-4089-b99f-cef0b751f464 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.162344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec029d-f600-4089-b99f-cef0b751f464 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-98ec029d-f600-4089-b99f-cef0b751f464 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:11.176819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:11.178182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 288/576] 10.222.0.83 () {56 vars in 1093 bytes} [Wed Aug 24 11:05:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:11.179447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec029d-f600-4089-b99f-cef0b751f464 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.239585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8aa6f-8f0a-42dc-8473-ed81cab8cfa8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.289418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:11.291272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254064e-aed3-4269-9def-fc4d358d47ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.291889 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254064e-aed3-4269-9def-fc4d358d47ef None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.292176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254064e-aed3-4269-9def-fc4d358d47ef None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.292446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254064e-aed3-4269-9def-fc4d358d47ef None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.293249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254064e-aed3-4269-9def-fc4d358d47ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:11.293556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254064e-aed3-4269-9def-fc4d358d47ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.294983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254064e-aed3-4269-9def-fc4d358d47ef None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.305099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 289/577] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:11.308981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254064e-aed3-4269-9def-fc4d358d47ef None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.317674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec029d-f600-4089-b99f-cef0b751f464 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.362965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a6759-6528-4632-895b-2fdea730f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.363708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a6759-6528-4632-895b-2fdea730f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.363985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a6759-6528-4632-895b-2fdea730f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.364263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a6759-6528-4632-895b-2fdea730f0a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.364978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a6759-6528-4632-895b-2fdea730f0a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:11.365318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a6759-6528-4632-895b-2fdea730f0a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.366899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a6759-6528-4632-895b-2fdea730f0a9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.379643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 289/578] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:11.380492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:11.380634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a6759-6528-4632-895b-2fdea730f0a9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.429077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254064e-aed3-4269-9def-fc4d358d47ef None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.465553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cfa9c-bef8-4c81-937a-45093a1cc823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.466184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cfa9c-bef8-4c81-937a-45093a1cc823 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.466455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cfa9c-bef8-4c81-937a-45093a1cc823 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.466820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cfa9c-bef8-4c81-937a-45093a1cc823 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.467579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cfa9c-bef8-4c81-937a-45093a1cc823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:11.467937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cfa9c-bef8-4c81-937a-45093a1cc823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.469933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cfa9c-bef8-4c81-937a-45093a1cc823 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.486071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 290/579] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:11.487010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cfa9c-bef8-4c81-937a-45093a1cc823 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.503511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a6759-6528-4632-895b-2fdea730f0a9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.542456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7081ed-ea78-4894-b0fe-f848b948cc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.543095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7081ed-ea78-4894-b0fe-f848b948cc86 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.543372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7081ed-ea78-4894-b0fe-f848b948cc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.544301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7081ed-ea78-4894-b0fe-f848b948cc86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.555901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7081ed-ea78-4894-b0fe-f848b948cc86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.557644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7081ed-ea78-4894-b0fe-f848b948cc86 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.559958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-cc7081ed-ea78-4894-b0fe-f848b948cc86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:11.573719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:11.575219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:11.576348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:11.576348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:11.576348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:11.576348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 290/580] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:11.577597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7081ed-ea78-4894-b0fe-f848b948cc86 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.597717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cfa9c-bef8-4c81-937a-45093a1cc823 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.623114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:11.635161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40a950-b3e7-4ec8-843e-cfacb9026b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.639315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40a950-b3e7-4ec8-843e-cfacb9026b48 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.639315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40a950-b3e7-4ec8-843e-cfacb9026b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.639315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40a950-b3e7-4ec8-843e-cfacb9026b48 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.639315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40a950-b3e7-4ec8-843e-cfacb9026b48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:11.639315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40a950-b3e7-4ec8-843e-cfacb9026b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.639315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40a950-b3e7-4ec8-843e-cfacb9026b48 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.657423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 291/581] 10.222.0.83 () {56 vars in 1133 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:11.658128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40a950-b3e7-4ec8-843e-cfacb9026b48 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.689904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7081ed-ea78-4894-b0fe-f848b948cc86 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.727830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000051fc-41fe-495d-943d-c2823aec3a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.728463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000051fc-41fe-495d-943d-c2823aec3a2c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.728751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000051fc-41fe-495d-943d-c2823aec3a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.729021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000051fc-41fe-495d-943d-c2823aec3a2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.740308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000051fc-41fe-495d-943d-c2823aec3a2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.741921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000051fc-41fe-495d-943d-c2823aec3a2c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-000051fc-41fe-495d-943d-c2823aec3a2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:11.756513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:11.757606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:11.758743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:11.758743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:11.758743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:11.758743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 291/582] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:11.758743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000051fc-41fe-495d-943d-c2823aec3a2c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.771155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40a950-b3e7-4ec8-843e-cfacb9026b48 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.806527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeab79-6d26-4cb9-a182-57d5103a3ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.807221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeab79-6d26-4cb9-a182-57d5103a3ebf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.807854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeab79-6d26-4cb9-a182-57d5103a3ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.808208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeab79-6d26-4cb9-a182-57d5103a3ebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.808940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eeab79-6d26-4cb9-a182-57d5103a3ebf None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=12eafcbf5c98425782138d5363db9a25) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:11.809216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:11.809339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eeab79-6d26-4cb9-a182-57d5103a3ebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.810764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eeab79-6d26-4cb9-a182-57d5103a3ebf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.816395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 292/583] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/role_assignments?user.id=12eafcbf5c98425782138d5363db9a25&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:11.822427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeab79-6d26-4cb9-a182-57d5103a3ebf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.874116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000051fc-41fe-495d-943d-c2823aec3a2c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.889219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:11.915696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835d130-bff3-4e51-b92b-86e354038120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.916346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835d130-bff3-4e51-b92b-86e354038120 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.916669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835d130-bff3-4e51-b92b-86e354038120 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:11.916908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835d130-bff3-4e51-b92b-86e354038120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:11.924627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835d130-bff3-4e51-b92b-86e354038120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:11.926523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835d130-bff3-4e51-b92b-86e354038120 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6835d130-bff3-4e51-b92b-86e354038120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:11.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:11.940670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:11.941563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:11.941563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:11.941563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:11.941563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:11.941563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:11.941563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:11.941563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:11.941563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:11.941563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 292/584] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:11.942083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835d130-bff3-4e51-b92b-86e354038120 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:11.960516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeab79-6d26-4cb9-a182-57d5103a3ebf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:11.998469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea850d-38a5-4bc0-8add-3f2b2f07261c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:11.999447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea850d-38a5-4bc0-8add-3f2b2f07261c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:11.999729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea850d-38a5-4bc0-8add-3f2b2f07261c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:12.000075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea850d-38a5-4bc0-8add-3f2b2f07261c None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:12.001077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea850d-38a5-4bc0-8add-3f2b2f07261c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:12.001385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea850d-38a5-4bc0-8add-3f2b2f07261c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:12.002805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea850d-38a5-4bc0-8add-3f2b2f07261c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:12.020524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 293/585] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:12.020524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea850d-38a5-4bc0-8add-3f2b2f07261c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:12.071920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835d130-bff3-4e51-b92b-86e354038120 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:12.080699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:12.108984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb34fb-6f0c-4951-8ce1-7b382abe8393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:12.109684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb34fb-6f0c-4951-8ce1-7b382abe8393 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:12.109968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb34fb-6f0c-4951-8ce1-7b382abe8393 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:12.110296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb34fb-6f0c-4951-8ce1-7b382abe8393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:12.121906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb34fb-6f0c-4951-8ce1-7b382abe8393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:12.123425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb34fb-6f0c-4951-8ce1-7b382abe8393 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-19fb34fb-6f0c-4951-8ce1-7b382abe8393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:12.139060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:12.140056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:12.141695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:12.141695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:12.141695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:12.141695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 293/586] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:12.141695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb34fb-6f0c-4951-8ce1-7b382abe8393 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:12.170987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea850d-38a5-4bc0-8add-3f2b2f07261c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:12.175081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:12.223561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a73ca-2176-4a1a-87e0-0223fe0510d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:12.224501 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a73ca-2176-4a1a-87e0-0223fe0510d9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:12.224781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a73ca-2176-4a1a-87e0-0223fe0510d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:12.225055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a73ca-2176-4a1a-87e0-0223fe0510d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:12.225710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a73ca-2176-4a1a-87e0-0223fe0510d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:12.226052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a73ca-2176-4a1a-87e0-0223fe0510d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:12.227498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a73ca-2176-4a1a-87e0-0223fe0510d9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:12.243997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 294/587] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:12.243997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a73ca-2176-4a1a-87e0-0223fe0510d9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:12.267297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb34fb-6f0c-4951-8ce1-7b382abe8393 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:12.317834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b833954-4b5e-4d62-b617-c4ccc0f033dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:12.318510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b833954-4b5e-4d62-b617-c4ccc0f033dd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:12.318791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b833954-4b5e-4d62-b617-c4ccc0f033dd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:12.319015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b833954-4b5e-4d62-b617-c4ccc0f033dd None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:12.319717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b833954-4b5e-4d62-b617-c4ccc0f033dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:12.320055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b833954-4b5e-4d62-b617-c4ccc0f033dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:12.321517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b833954-4b5e-4d62-b617-c4ccc0f033dd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:12.335715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 294/588] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:12.336571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b833954-4b5e-4d62-b617-c4ccc0f033dd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:12.362572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:12.370856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a73ca-2176-4a1a-87e0-0223fe0510d9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:12.412396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbb32c-1a54-4e26-9d0f-707774419d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:12.414041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbb32c-1a54-4e26-9d0f-707774419d60 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:12.414477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbb32c-1a54-4e26-9d0f-707774419d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:12.414920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbb32c-1a54-4e26-9d0f-707774419d60 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:12.424793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbb32c-1a54-4e26-9d0f-707774419d60 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:12.426283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbb32c-1a54-4e26-9d0f-707774419d60 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8dbbb32c-1a54-4e26-9d0f-707774419d60 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:12.439106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:12.440105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 295/589] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:12.441169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbb32c-1a54-4e26-9d0f-707774419d60 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:12.465373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b833954-4b5e-4d62-b617-c4ccc0f033dd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:12.471623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:12.505997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d4b6d-c3ae-49ea-92e9-dfb11ecd4c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:12.506754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d4b6d-c3ae-49ea-92e9-dfb11ecd4c99 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:12.507042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d4b6d-c3ae-49ea-92e9-dfb11ecd4c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:12.507320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d4b6d-c3ae-49ea-92e9-dfb11ecd4c99 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:12.507987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d4b6d-c3ae-49ea-92e9-dfb11ecd4c99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:12.508295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d4b6d-c3ae-49ea-92e9-dfb11ecd4c99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:12.509700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d4b6d-c3ae-49ea-92e9-dfb11ecd4c99 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:12.524501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 295/590] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:12.525400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d4b6d-c3ae-49ea-92e9-dfb11ecd4c99 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:12.567637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbb32c-1a54-4e26-9d0f-707774419d60 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:12.608702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abece3-465f-464c-a5bd-c6baee77ddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:12.609801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abece3-465f-464c-a5bd-c6baee77ddbb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:12.610113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abece3-465f-464c-a5bd-c6baee77ddbb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:12.610389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abece3-465f-464c-a5bd-c6baee77ddbb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:12.612437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:12.619810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abece3-465f-464c-a5bd-c6baee77ddbb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:12.621269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abece3-465f-464c-a5bd-c6baee77ddbb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-12abece3-465f-464c-a5bd-c6baee77ddbb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:12.633274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:12.634269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 296/591] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:12.635203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abece3-465f-464c-a5bd-c6baee77ddbb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:12.638488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d4b6d-c3ae-49ea-92e9-dfb11ecd4c99 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:12.683074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411b7d8-2377-4ac1-ba8a-18f08b7a255c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:12.683990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411b7d8-2377-4ac1-ba8a-18f08b7a255c None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:12.684291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411b7d8-2377-4ac1-ba8a-18f08b7a255c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:12.684789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411b7d8-2377-4ac1-ba8a-18f08b7a255c None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/f1f214c2b77d4cbab2f15122ba21bf38/roles/7612e1bae5ad4bda9f0675ad732646b4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:12.708684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411b7d8-2377-4ac1-ba8a-18f08b7a255c None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=f1f214c2b77d4cbab2f15122ba21bf38, role_id=7612e1bae5ad4bda9f0675ad732646b4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:12.711246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411b7d8-2377-4ac1-ba8a-18f08b7a255c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:12.752222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:12.769388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abece3-465f-464c-a5bd-c6baee77ddbb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:12.804216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c88db1-6d12-47ce-a016-eb6e235c3ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:12.804853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c88db1-6d12-47ce-a016-eb6e235c3ecf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:12.805173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c88db1-6d12-47ce-a016-eb6e235c3ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:12.805449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c88db1-6d12-47ce-a016-eb6e235c3ecf None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:12.806266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c88db1-6d12-47ce-a016-eb6e235c3ecf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:12.806573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c88db1-6d12-47ce-a016-eb6e235c3ecf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:12.808317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c88db1-6d12-47ce-a016-eb6e235c3ecf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:12.832710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 296/592] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:05:12 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/f1f214c2b77d4cbab2f15122ba21bf38/roles/7612e1bae5ad4bda9f0675ad732646b4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:05:12.832710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411b7d8-2377-4ac1-ba8a-18f08b7a255c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:12.835813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 297/593] 10.222.0.83 () {56 vars in 1137 bytes} [Wed Aug 24 11:05:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:12.837749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c88db1-6d12-47ce-a016-eb6e235c3ecf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:12.983951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411b7d8-2377-4ac1-ba8a-18f08b7a255c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:12.987786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c88db1-6d12-47ce-a016-eb6e235c3ecf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:13.023646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:13.030840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ed278-e30c-4b0e-8558-8c1df7c77867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.031428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ed278-e30c-4b0e-8558-8c1df7c77867 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.039681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ed278-e30c-4b0e-8558-8c1df7c77867 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.039681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ed278-e30c-4b0e-8558-8c1df7c77867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.039924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357187a-be88-416b-92de-c716694b4faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.039924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357187a-be88-416b-92de-c716694b4faf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.039924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357187a-be88-416b-92de-c716694b4faf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.039924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357187a-be88-416b-92de-c716694b4faf None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.040268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357187a-be88-416b-92de-c716694b4faf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:13.040268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357187a-be88-416b-92de-c716694b4faf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:13.040268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357187a-be88-416b-92de-c716694b4faf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:13.040511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ed278-e30c-4b0e-8558-8c1df7c77867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:13.041121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ed278-e30c-4b0e-8558-8c1df7c77867 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-982ed278-e30c-4b0e-8558-8c1df7c77867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:13.055006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:13.056170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:13.057275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:13.057275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:13.057275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:13.057275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:13.057275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:13.057275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:13.057275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:13.057275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:13.057275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 297/594] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:13.058185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ed278-e30c-4b0e-8558-8c1df7c77867 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:13.067349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:13.067681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 298/595] 10.222.0.83 () {56 vars in 1139 bytes} [Wed Aug 24 11:05:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:13.069644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357187a-be88-416b-92de-c716694b4faf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:13.190765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ed278-e30c-4b0e-8558-8c1df7c77867 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:13.200397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357187a-be88-416b-92de-c716694b4faf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:13.227465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223021d2-1d69-4ef2-8343-7e7900a2d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.228259 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223021d2-1d69-4ef2-8343-7e7900a2d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.228640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223021d2-1d69-4ef2-8343-7e7900a2d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.229015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223021d2-1d69-4ef2-8343-7e7900a2d0e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.241578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223021d2-1d69-4ef2-8343-7e7900a2d0e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:13.243511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223021d2-1d69-4ef2-8343-7e7900a2d0e2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:13.245302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80124e93-f857-4c28-a658-1f67dedca34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.246115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80124e93-f857-4c28-a658-1f67dedca34b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.246391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80124e93-f857-4c28-a658-1f67dedca34b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.246661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80124e93-f857-4c28-a658-1f67dedca34b None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.247767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80124e93-f857-4c28-a658-1f67dedca34b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:13.248074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80124e93-f857-4c28-a658-1f67dedca34b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:13.249512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80124e93-f857-4c28-a658-1f67dedca34b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-223021d2-1d69-4ef2-8343-7e7900a2d0e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:13.260087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:13.261854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:13.263991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:13.263991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:13.263991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:13.269006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 299/597] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:13.269006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80124e93-f857-4c28-a658-1f67dedca34b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:13.269263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 298/597] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:13.272834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223021d2-1d69-4ef2-8343-7e7900a2d0e2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:13.333316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:13.349431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:13.401910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80124e93-f857-4c28-a658-1f67dedca34b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:13.410114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223021d2-1d69-4ef2-8343-7e7900a2d0e2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:13.445794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79821af0-386b-4031-ac4c-b310b49531e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.446936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79821af0-386b-4031-ac4c-b310b49531e3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.447715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79821af0-386b-4031-ac4c-b310b49531e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.448449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79821af0-386b-4031-ac4c-b310b49531e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.456066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ef142-254a-499b-b7e4-cf345ad9fc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.456686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ef142-254a-499b-b7e4-cf345ad9fc18 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.456969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ef142-254a-499b-b7e4-cf345ad9fc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.457246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ef142-254a-499b-b7e4-cf345ad9fc18 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.457941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ef142-254a-499b-b7e4-cf345ad9fc18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:13.458253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ef142-254a-499b-b7e4-cf345ad9fc18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:13.459877 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ef142-254a-499b-b7e4-cf345ad9fc18 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:13.465215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79821af0-386b-4031-ac4c-b310b49531e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:13.467115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79821af0-386b-4031-ac4c-b310b49531e3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:13.480809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 299/598] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:13.481808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ef142-254a-499b-b7e4-cf345ad9fc18 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-79821af0-386b-4031-ac4c-b310b49531e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:13.488151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:13.489176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:13.490112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:13.490112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:13.490112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:13.492011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 300/599] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:13.493663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79821af0-386b-4031-ac4c-b310b49531e3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:13.596008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:13.600057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ef142-254a-499b-b7e4-cf345ad9fc18 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:13.626639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79821af0-386b-4031-ac4c-b310b49531e3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:13.639911 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c93028-35f9-44ba-beb5-b8f3cf18958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.640579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c93028-35f9-44ba-beb5-b8f3cf18958a None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.641632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c93028-35f9-44ba-beb5-b8f3cf18958a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.642073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c93028-35f9-44ba-beb5-b8f3cf18958a None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/32a28f9fb3134f33819e53e0c72c63b4/roles/7612e1bae5ad4bda9f0675ad732646b4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.643973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:13.662109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c93028-35f9-44ba-beb5-b8f3cf18958a None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=32a28f9fb3134f33819e53e0c72c63b4, role_id=7612e1bae5ad4bda9f0675ad732646b4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:13.664456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c93028-35f9-44ba-beb5-b8f3cf18958a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:13.667930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd923e-c12f-40b3-a0eb-112d8c5e429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.668883 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd923e-c12f-40b3-a0eb-112d8c5e429d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.669452 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd923e-c12f-40b3-a0eb-112d8c5e429d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.670276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd923e-c12f-40b3-a0eb-112d8c5e429d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.677825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd923e-c12f-40b3-a0eb-112d8c5e429d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:13.679561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd923e-c12f-40b3-a0eb-112d8c5e429d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-43fd923e-c12f-40b3-a0eb-112d8c5e429d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:13.692931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:13.696733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 301/600] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:13.697652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd923e-c12f-40b3-a0eb-112d8c5e429d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:13.790719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 300/601] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:05:13 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/32a28f9fb3134f33819e53e0c72c63b4/roles/7612e1bae5ad4bda9f0675ad732646b4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:05:13.793066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c93028-35f9-44ba-beb5-b8f3cf18958a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:13.839563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd923e-c12f-40b3-a0eb-112d8c5e429d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:13.891881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b31a2-9d32-4325-af17-845bf412c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.892517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b31a2-9d32-4325-af17-845bf412c580 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.892836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b31a2-9d32-4325-af17-845bf412c580 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.893115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b31a2-9d32-4325-af17-845bf412c580 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.894101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b31a2-9d32-4325-af17-845bf412c580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:13.894415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b31a2-9d32-4325-af17-845bf412c580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:13.896786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b31a2-9d32-4325-af17-845bf412c580 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:13.915518 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 302/602] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:13.917997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f6a20-f6fa-4b67-92fa-a7e75e3f1764 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.918284 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f6a20-f6fa-4b67-92fa-a7e75e3f1764 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.918558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f6a20-f6fa-4b67-92fa-a7e75e3f1764 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:13.920274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 303/603] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:13.921076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f6a20-f6fa-4b67-92fa-a7e75e3f1764 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:13.952465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c93028-35f9-44ba-beb5-b8f3cf18958a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:13.960758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:13.998444 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8ab35-36ac-40ab-ad36-a94bca66abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:13.999068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8ab35-36ac-40ab-ad36-a94bca66abdf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:13.999347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8ab35-36ac-40ab-ad36-a94bca66abdf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:13.999620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8ab35-36ac-40ab-ad36-a94bca66abdf None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:14.000978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8ab35-36ac-40ab-ad36-a94bca66abdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:14.001200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8ab35-36ac-40ab-ad36-a94bca66abdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:14.002690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8ab35-36ac-40ab-ad36-a94bca66abdf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:14.018110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 301/604] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:14.019520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6e87b-a2ad-4963-9dc6-551cb6822a86 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:14.019809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6e87b-a2ad-4963-9dc6-551cb6822a86 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:14.020085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6e87b-a2ad-4963-9dc6-551cb6822a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:14.040866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-93c6e87b-a2ad-4963-9dc6-551cb6822a86 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:14.042071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-93c6e87b-a2ad-4963-9dc6-551cb6822a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:14.046578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:14.056952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f6a20-f6fa-4b67-92fa-a7e75e3f1764 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:14.099511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dc6ee-5c82-4934-afac-3682290e5517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:14.100303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dc6ee-5c82-4934-afac-3682290e5517 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:14.100587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dc6ee-5c82-4934-afac-3682290e5517 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:14.101042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dc6ee-5c82-4934-afac-3682290e5517 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:14.108925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dc6ee-5c82-4934-afac-3682290e5517 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:14.110802 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dc6ee-5c82-4934-afac-3682290e5517 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-ca4dc6ee-5c82-4934-afac-3682290e5517 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:14.123973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:14.124916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:14.126442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:14.126442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:14.126442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:14.126442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:14.126442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:14.126442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:14.126442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:14.126442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:14.126442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 304/606] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:14.126923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 302/606] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:14.127017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dc6ee-5c82-4934-afac-3682290e5517 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:14.137812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6e87b-a2ad-4963-9dc6-551cb6822a86 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:14.285641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:14.300270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dc6ee-5c82-4934-afac-3682290e5517 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:14.314284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6e87b-a2ad-4963-9dc6-551cb6822a86 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:14.333728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce6f13-1488-4a02-b2fb-7c8d366e1278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:14.334630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce6f13-1488-4a02-b2fb-7c8d366e1278 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:14.335058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce6f13-1488-4a02-b2fb-7c8d366e1278 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:14.336512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce6f13-1488-4a02-b2fb-7c8d366e1278 None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/f9dd6f187dd84f04866da64732b242d7/roles/7612e1bae5ad4bda9f0675ad732646b4` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:14.351926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99e963-039b-494a-af3e-d82a64b02ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:14.352681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99e963-039b-494a-af3e-d82a64b02ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:14.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99e963-039b-494a-af3e-d82a64b02ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:14.353437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99e963-039b-494a-af3e-d82a64b02ef7 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:14.354288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99e963-039b-494a-af3e-d82a64b02ef7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:14.354697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99e963-039b-494a-af3e-d82a64b02ef7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:14.355080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce6f13-1488-4a02-b2fb-7c8d366e1278 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=f9dd6f187dd84f04866da64732b242d7, role_id=7612e1bae5ad4bda9f0675ad732646b4)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:14.356401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99e963-039b-494a-af3e-d82a64b02ef7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:14.356832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce6f13-1488-4a02-b2fb-7c8d366e1278 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:14.371491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:14.371836 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 303/607] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:14.376965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99e963-039b-494a-af3e-d82a64b02ef7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:14.478338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 305/608] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:05:14 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/f9dd6f187dd84f04866da64732b242d7/roles/7612e1bae5ad4bda9f0675ad732646b4 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:05:14.487300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce6f13-1488-4a02-b2fb-7c8d366e1278 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:14.495786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99e963-039b-494a-af3e-d82a64b02ef7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:14.540405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dd9a1-2694-49c8-96f0-b7a96d66f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:14.541187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dd9a1-2694-49c8-96f0-b7a96d66f238 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:14.541474 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dd9a1-2694-49c8-96f0-b7a96d66f238 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:14.541754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dd9a1-2694-49c8-96f0-b7a96d66f238 None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/b0145242ef784168bada8348ccd95f22/roles/7612e1bae5ad4bda9f0675ad732646b4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:14.563661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326dd9a1-2694-49c8-96f0-b7a96d66f238 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=b0145242ef784168bada8348ccd95f22, role_id=7612e1bae5ad4bda9f0675ad732646b4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:14.565331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326dd9a1-2694-49c8-96f0-b7a96d66f238 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:14.655149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 304/609] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:05:14 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/b0145242ef784168bada8348ccd95f22/roles/7612e1bae5ad4bda9f0675ad732646b4 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:05:14.660141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce6f13-1488-4a02-b2fb-7c8d366e1278 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:14.703101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbfbff-2cba-4dc7-abd2-0dbb3c2d1adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:14.703840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbfbff-2cba-4dc7-abd2-0dbb3c2d1adb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:14.704123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbfbff-2cba-4dc7-abd2-0dbb3c2d1adb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:14.704391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbfbff-2cba-4dc7-abd2-0dbb3c2d1adb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:14.717030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbfbff-2cba-4dc7-abd2-0dbb3c2d1adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-81fbfbff-2cba-4dc7-abd2-0dbb3c2d1adb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:14.722137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:14.723130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:14.723941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 306/610] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:14.729659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dd9a1-2694-49c8-96f0-b7a96d66f238 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:14.802483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:14.886598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dd9a1-2694-49c8-96f0-b7a96d66f238 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:14.937468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f3aad-5bb7-4ff5-94ef-272de1c41531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:14.938684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f3aad-5bb7-4ff5-94ef-272de1c41531 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:14.938971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f3aad-5bb7-4ff5-94ef-272de1c41531 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:14.939253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f3aad-5bb7-4ff5-94ef-272de1c41531 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:14.939984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f3aad-5bb7-4ff5-94ef-272de1c41531 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:14.940295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f3aad-5bb7-4ff5-94ef-272de1c41531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:14.941728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f3aad-5bb7-4ff5-94ef-272de1c41531 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:14.951514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 305/611] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:14.957023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbfbff-2cba-4dc7-abd2-0dbb3c2d1adb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:14.964760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:15.003809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea822698-e241-4dfc-ae76-0142b7b39c47 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:15.004465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea822698-e241-4dfc-ae76-0142b7b39c47 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:15.004951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea822698-e241-4dfc-ae76-0142b7b39c47 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:15.006924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 306/612] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:15.013177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e608ca-8148-41eb-b89a-4504a9292e68 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:15.013773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e608ca-8148-41eb-b89a-4504a9292e68 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:15.014253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e608ca-8148-41eb-b89a-4504a9292e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:15.034950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-19e608ca-8148-41eb-b89a-4504a9292e68 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:15.036528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-19e608ca-8148-41eb-b89a-4504a9292e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:15.080932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbfbff-2cba-4dc7-abd2-0dbb3c2d1adb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:15.123767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226533d4-216d-406f-be81-06d290c6e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:15.124405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226533d4-216d-406f-be81-06d290c6e623 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:15.124682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226533d4-216d-406f-be81-06d290c6e623 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:15.124942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226533d4-216d-406f-be81-06d290c6e623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:15.141791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 307/613] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:05:15.143828 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226533d4-216d-406f-be81-06d290c6e623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:15.145295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226533d4-216d-406f-be81-06d290c6e623 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-226533d4-216d-406f-be81-06d290c6e623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:15.177748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:15.178814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:15.179705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:15.179705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:15.179705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:15.179705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 307/614] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:15.185904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e608ca-8148-41eb-b89a-4504a9292e68 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:15.256667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:15.339909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e608ca-8148-41eb-b89a-4504a9292e68 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:15.395921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8967cf2-0619-40c8-991c-a9949f9f5cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:15.396542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8967cf2-0619-40c8-991c-a9949f9f5cec None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:15.396780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8967cf2-0619-40c8-991c-a9949f9f5cec None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:15.396983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8967cf2-0619-40c8-991c-a9949f9f5cec None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:15.397657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8967cf2-0619-40c8-991c-a9949f9f5cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:15.397990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8967cf2-0619-40c8-991c-a9949f9f5cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:15.399386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8967cf2-0619-40c8-991c-a9949f9f5cec None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:15.419275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 308/615] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:15.426069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226533d4-216d-406f-be81-06d290c6e623 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:15.507191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:15.570713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226533d4-216d-406f-be81-06d290c6e623 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:15.619702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f4bbe-3485-4314-8a36-5563d3d8c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:15.620337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f4bbe-3485-4314-8a36-5563d3d8c66f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:15.620620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f4bbe-3485-4314-8a36-5563d3d8c66f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:15.620897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f4bbe-3485-4314-8a36-5563d3d8c66f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:15.629993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8967cf2-0619-40c8-991c-a9949f9f5cec None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:15.631234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f4bbe-3485-4314-8a36-5563d3d8c66f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:15.632890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f4bbe-3485-4314-8a36-5563d3d8c66f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8a4f4bbe-3485-4314-8a36-5563d3d8c66f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:15.652538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:15.653533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:15.654554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:15.661049 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 308/616] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:15.664134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f4bbe-3485-4314-8a36-5563d3d8c66f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:15.849796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8967cf2-0619-40c8-991c-a9949f9f5cec None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:15.856767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f4bbe-3485-4314-8a36-5563d3d8c66f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:15.858631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:15.899239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:15.899775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9af5e-354e-4ccf-b9ad-f3e436e8de38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:15.900436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9af5e-354e-4ccf-b9ad-f3e436e8de38 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:15.900719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9af5e-354e-4ccf-b9ad-f3e436e8de38 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:15.900992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9af5e-354e-4ccf-b9ad-f3e436e8de38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:15.928299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9af5e-354e-4ccf-b9ad-f3e436e8de38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-4cd9af5e-354e-4ccf-b9ad-f3e436e8de38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:15.934068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:15.935024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:15.935848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 309/617] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:15.942833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9af5e-354e-4ccf-b9ad-f3e436e8de38 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:15.948796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563f879-8d89-4871-9d85-8506247f50f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:15.950387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563f879-8d89-4871-9d85-8506247f50f9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:15.951149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563f879-8d89-4871-9d85-8506247f50f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:15.951830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563f879-8d89-4871-9d85-8506247f50f9 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:15.952795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563f879-8d89-4871-9d85-8506247f50f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:15.957560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563f879-8d89-4871-9d85-8506247f50f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:15.959303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563f879-8d89-4871-9d85-8506247f50f9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:15.990736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 309/618] 10.222.0.83 () {56 vars in 1135 bytes} [Wed Aug 24 11:05:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:15.997193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563f879-8d89-4871-9d85-8506247f50f9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:16.148776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9af5e-354e-4ccf-b9ad-f3e436e8de38 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:16.162508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563f879-8d89-4871-9d85-8506247f50f9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:16.190980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b7fe0-4734-4333-bf64-c90c2733c525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:16.191518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b7fe0-4734-4333-bf64-c90c2733c525 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.191821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b7fe0-4734-4333-bf64-c90c2733c525 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.192176 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b7fe0-4734-4333-bf64-c90c2733c525 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.192993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b7fe0-4734-4333-bf64-c90c2733c525 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:16.193307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b7fe0-4734-4333-bf64-c90c2733c525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:16.194937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b7fe0-4734-4333-bf64-c90c2733c525 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:16.203312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a803c13-8790-4436-9539-16943110b3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:16.204104 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a803c13-8790-4436-9539-16943110b3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.204388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a803c13-8790-4436-9539-16943110b3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.204661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a803c13-8790-4436-9539-16943110b3a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.205324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 310/619] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:16.206614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470b32b-e18b-4d73-8a7e-664f48798663 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.206897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470b32b-e18b-4d73-8a7e-664f48798663 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.207168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470b32b-e18b-4d73-8a7e-664f48798663 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.208363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 311/620] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:16.214983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470b32b-e18b-4d73-8a7e-664f48798663 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:16.221185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a803c13-8790-4436-9539-16943110b3a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:16.222684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a803c13-8790-4436-9539-16943110b3a0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4a803c13-8790-4436-9539-16943110b3a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:16.239421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:16.240418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:16.241305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:16.241305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:16.241305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:16.243043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:16.245368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 310/621] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:16.246769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfeb3a-0385-4676-81ad-36acf43fa6c5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.247060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfeb3a-0385-4676-81ad-36acf43fa6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.247340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfeb3a-0385-4676-81ad-36acf43fa6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.266547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b2bfeb3a-0385-4676-81ad-36acf43fa6c5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:16.267769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b2bfeb3a-0385-4676-81ad-36acf43fa6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:16.290009 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:16.328502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470b32b-e18b-4d73-8a7e-664f48798663 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:16.353985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 311/622] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:16.353985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfeb3a-0385-4676-81ad-36acf43fa6c5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:16.401242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2227e7c-7440-40ab-a1b9-42ffe7cd9ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:16.402713 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2227e7c-7440-40ab-a1b9-42ffe7cd9ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.403197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2227e7c-7440-40ab-a1b9-42ffe7cd9ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.403712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2227e7c-7440-40ab-a1b9-42ffe7cd9ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.431461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2227e7c-7440-40ab-a1b9-42ffe7cd9ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:16.433584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2227e7c-7440-40ab-a1b9-42ffe7cd9ff2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-b2227e7c-7440-40ab-a1b9-42ffe7cd9ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:16.464895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:16.466135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:16.467089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:16.467089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:16.467089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:16.467275 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfeb3a-0385-4676-81ad-36acf43fa6c5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:16.474375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 312/623] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:16.475605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2227e7c-7440-40ab-a1b9-42ffe7cd9ff2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:16.513590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216ae33-af9d-40b9-98c9-bb388ba0feb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:16.514273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216ae33-af9d-40b9-98c9-bb388ba0feb8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.514555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216ae33-af9d-40b9-98c9-bb388ba0feb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.514830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216ae33-af9d-40b9-98c9-bb388ba0feb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.515491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216ae33-af9d-40b9-98c9-bb388ba0feb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:16.515824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216ae33-af9d-40b9-98c9-bb388ba0feb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:16.517537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216ae33-af9d-40b9-98c9-bb388ba0feb8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:16.539863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 312/624] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:16.545383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216ae33-af9d-40b9-98c9-bb388ba0feb8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:16.570308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:16.651708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:16.655095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216ae33-af9d-40b9-98c9-bb388ba0feb8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:16.695346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d74116-ea17-4736-9a6d-43f830a5e9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:16.695987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d74116-ea17-4736-9a6d-43f830a5e9bf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.696244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d74116-ea17-4736-9a6d-43f830a5e9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.696534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d74116-ea17-4736-9a6d-43f830a5e9bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.703957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d74116-ea17-4736-9a6d-43f830a5e9bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:16.705432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d74116-ea17-4736-9a6d-43f830a5e9bf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a2d74116-ea17-4736-9a6d-43f830a5e9bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:16.717595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:16.718570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:16.719991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:16.719991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:16.719991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:16.719991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:16.719991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:16.719991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:16.719991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:16.719991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:16.719991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 313/625] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:16.724156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2227e7c-7440-40ab-a1b9-42ffe7cd9ff2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:16.725161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d74116-ea17-4736-9a6d-43f830a5e9bf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:16.780540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4686b5b-7a08-4f7a-91fd-8a1bcc2929ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:16.781188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4686b5b-7a08-4f7a-91fd-8a1bcc2929ba None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.782064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4686b5b-7a08-4f7a-91fd-8a1bcc2929ba None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.782338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4686b5b-7a08-4f7a-91fd-8a1bcc2929ba None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.782995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4686b5b-7a08-4f7a-91fd-8a1bcc2929ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:16.783301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4686b5b-7a08-4f7a-91fd-8a1bcc2929ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:16.784703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4686b5b-7a08-4f7a-91fd-8a1bcc2929ba None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:16.798117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 313/626] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:16.798985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4686b5b-7a08-4f7a-91fd-8a1bcc2929ba None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:16.821470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:16.849981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d74116-ea17-4736-9a6d-43f830a5e9bf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:16.893795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213297fa-3c60-4486-9d80-510806d0e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:16.894447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213297fa-3c60-4486-9d80-510806d0e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.894731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213297fa-3c60-4486-9d80-510806d0e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.895067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213297fa-3c60-4486-9d80-510806d0e9a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.895777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213297fa-3c60-4486-9d80-510806d0e9a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:16.896086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213297fa-3c60-4486-9d80-510806d0e9a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:16.897707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213297fa-3c60-4486-9d80-510806d0e9a1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:16.913144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 314/627] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:16.914143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213297fa-3c60-4486-9d80-510806d0e9a1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:16.929684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4686b5b-7a08-4f7a-91fd-8a1bcc2929ba None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:16.969201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53633c-7813-423e-ac90-986fba68dd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:16.971814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53633c-7813-423e-ac90-986fba68dd4b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.971814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53633c-7813-423e-ac90-986fba68dd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.971814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53633c-7813-423e-ac90-986fba68dd4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.975013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:16.984383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53633c-7813-423e-ac90-986fba68dd4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ee53633c-7813-423e-ac90-986fba68dd4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:16.989783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:16.990763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:16.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 314/628] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:16.994299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5f59c-639e-45af-ab3d-c1e4897bba86 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:16.994299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5f59c-639e-45af-ab3d-c1e4897bba86 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:16.994565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5f59c-639e-45af-ab3d-c1e4897bba86 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:16.995992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 315/629] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:16.996561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5f59c-639e-45af-ab3d-c1e4897bba86 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:17.036483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213297fa-3c60-4486-9d80-510806d0e9a1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:17.077898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4495bb-d2f8-43ca-bb13-103baadd9f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:17.078557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4495bb-d2f8-43ca-bb13-103baadd9f3d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.078834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4495bb-d2f8-43ca-bb13-103baadd9f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.079103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4495bb-d2f8-43ca-bb13-103baadd9f3d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.081122 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:17.087197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4495bb-d2f8-43ca-bb13-103baadd9f3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:17.088687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4495bb-d2f8-43ca-bb13-103baadd9f3d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0b4495bb-d2f8-43ca-bb13-103baadd9f3d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:17.101223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:17.111587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 315/630] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:17.112526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4495bb-d2f8-43ca-bb13-103baadd9f3d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:17.121747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5f59c-639e-45af-ab3d-c1e4897bba86 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:17.160138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba939d-3a65-4804-9065-b9a337d54e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:17.160941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba939d-3a65-4804-9065-b9a337d54e54 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.161291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba939d-3a65-4804-9065-b9a337d54e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.161703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba939d-3a65-4804-9065-b9a337d54e54 None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/12eafcbf5c98425782138d5363db9a25/roles/7612e1bae5ad4bda9f0675ad732646b4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.178907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba939d-3a65-4804-9065-b9a337d54e54 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=12eafcbf5c98425782138d5363db9a25, role_id=7612e1bae5ad4bda9f0675ad732646b4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:17.180426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba939d-3a65-4804-9065-b9a337d54e54 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:17.217019 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4495bb-d2f8-43ca-bb13-103baadd9f3d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:17.252521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863b804-89a4-4a86-bb00-670d924742a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:17.253033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863b804-89a4-4a86-bb00-670d924742a2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.253312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863b804-89a4-4a86-bb00-670d924742a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.253580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863b804-89a4-4a86-bb00-670d924742a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.254295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863b804-89a4-4a86-bb00-670d924742a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:17.254599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863b804-89a4-4a86-bb00-670d924742a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:17.256036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863b804-89a4-4a86-bb00-670d924742a2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:17.258353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 316/631] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:05:16 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/12eafcbf5c98425782138d5363db9a25/roles/7612e1bae5ad4bda9f0675ad732646b4 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:05:17.259558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e03416-e2c2-4d1d-9a56-65b2e9860acd None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.259975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e03416-e2c2-4d1d-9a56-65b2e9860acd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.260343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e03416-e2c2-4d1d-9a56-65b2e9860acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.272905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 316/632] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:17.274722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863b804-89a4-4a86-bb00-670d924742a2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:17.294659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b4e03416-e2c2-4d1d-9a56-65b2e9860acd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:17.297575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b4e03416-e2c2-4d1d-9a56-65b2e9860acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:17.367975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:17.381213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:17.419177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 317/633] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:17.420752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629ac21-44f1-47b7-8a51-d35017160f6f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.421041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629ac21-44f1-47b7-8a51-d35017160f6f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.422707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629ac21-44f1-47b7-8a51-d35017160f6f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.424295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 318/634] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:17.427367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629ac21-44f1-47b7-8a51-d35017160f6f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:17.428369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863b804-89a4-4a86-bb00-670d924742a2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:17.464574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4261e-0cef-4a77-9b2d-9459873ffdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:17.465436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4261e-0cef-4a77-9b2d-9459873ffdda None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.465718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4261e-0cef-4a77-9b2d-9459873ffdda None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.466146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4261e-0cef-4a77-9b2d-9459873ffdda None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.466992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4261e-0cef-4a77-9b2d-9459873ffdda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:17.467400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4261e-0cef-4a77-9b2d-9459873ffdda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:17.469840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4261e-0cef-4a77-9b2d-9459873ffdda None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:17.494818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 317/635] 10.222.0.83 () {56 vars in 1137 bytes} [Wed Aug 24 11:05:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:17.494818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33918437-fc61-4699-b74d-6593534453b3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.494818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33918437-fc61-4699-b74d-6593534453b3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.494818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33918437-fc61-4699-b74d-6593534453b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.511586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-33918437-fc61-4699-b74d-6593534453b3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:17.512962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-33918437-fc61-4699-b74d-6593534453b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:17.580900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629ac21-44f1-47b7-8a51-d35017160f6f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:17.595993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 318/636] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:17.597031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33918437-fc61-4699-b74d-6593534453b3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:17.632309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3ecd6-d4d5-4ea6-8b15-2e8c19ae638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:17.633484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3ecd6-d4d5-4ea6-8b15-2e8c19ae638a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.633972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3ecd6-d4d5-4ea6-8b15-2e8c19ae638a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.634827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3ecd6-d4d5-4ea6-8b15-2e8c19ae638a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.649783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3ecd6-d4d5-4ea6-8b15-2e8c19ae638a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:17.651528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:17.653387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3ecd6-d4d5-4ea6-8b15-2e8c19ae638a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-c7b3ecd6-d4d5-4ea6-8b15-2e8c19ae638a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:17.674188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:17.675382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:17.676485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:17.676485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:17.676485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:17.678525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 319/637] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:17.684747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3ecd6-d4d5-4ea6-8b15-2e8c19ae638a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:17.720678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:17.723164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33918437-fc61-4699-b74d-6593534453b3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:17.769630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186b3aa-311c-48f3-be2d-f29e819c8066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:17.770506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186b3aa-311c-48f3-be2d-f29e819c8066 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.770957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186b3aa-311c-48f3-be2d-f29e819c8066 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.771418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186b3aa-311c-48f3-be2d-f29e819c8066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.803970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3ecd6-d4d5-4ea6-8b15-2e8c19ae638a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:17.846276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ba366-6eb5-4db5-aa54-3ff4500635ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:17.847058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ba366-6eb5-4db5-aa54-3ff4500635ee None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:17.847441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ba366-6eb5-4db5-aa54-3ff4500635ee None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:17.847815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ba366-6eb5-4db5-aa54-3ff4500635ee None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:17.848620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ba366-6eb5-4db5-aa54-3ff4500635ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:17.849025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ba366-6eb5-4db5-aa54-3ff4500635ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:17.850625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ba366-6eb5-4db5-aa54-3ff4500635ee None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:17.854940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186b3aa-311c-48f3-be2d-f29e819c8066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:17.856397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186b3aa-311c-48f3-be2d-f29e819c8066 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:17.864744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 320/638] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:17.865826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ba366-6eb5-4db5-aa54-3ff4500635ee None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:17.919172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:17.965932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:17.970076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 319/639] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:17.971152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186b3aa-311c-48f3-be2d-f29e819c8066 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:17.986704 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ba366-6eb5-4db5-aa54-3ff4500635ee None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.030366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab51dd-e413-49e1-bbcf-b8b26ac7c7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.032246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab51dd-e413-49e1-bbcf-b8b26ac7c7df None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.032754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab51dd-e413-49e1-bbcf-b8b26ac7c7df None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.033153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab51dd-e413-49e1-bbcf-b8b26ac7c7df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.047763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab51dd-e413-49e1-bbcf-b8b26ac7c7df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-19ab51dd-e413-49e1-bbcf-b8b26ac7c7df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:18.053840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:18.055533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 321/640] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:18.056618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab51dd-e413-49e1-bbcf-b8b26ac7c7df None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:18.097929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186b3aa-311c-48f3-be2d-f29e819c8066 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.138843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ff33c-3f04-4574-8eb3-9a722a49e0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.139618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ff33c-3f04-4574-8eb3-9a722a49e0ea None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.139896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ff33c-3f04-4574-8eb3-9a722a49e0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.141409 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ff33c-3f04-4574-8eb3-9a722a49e0ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.148622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ff33c-3f04-4574-8eb3-9a722a49e0ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:18.150211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ff33c-3f04-4574-8eb3-9a722a49e0ea None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-5f8ff33c-3f04-4574-8eb3-9a722a49e0ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:18.164794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:18.165924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 320/641] 10.222.0.83 () {56 vars in 1093 bytes} [Wed Aug 24 11:05:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:18.166892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ff33c-3f04-4574-8eb3-9a722a49e0ea None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:18.175668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:18.190557 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab51dd-e413-49e1-bbcf-b8b26ac7c7df None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.233369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea74f7c-d038-4892-b822-251889c7d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.234100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea74f7c-d038-4892-b822-251889c7d8ba None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.234401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea74f7c-d038-4892-b822-251889c7d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.234671 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea74f7c-d038-4892-b822-251889c7d8ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.245920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea74f7c-d038-4892-b822-251889c7d8ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-9ea74f7c-d038-4892-b822-251889c7d8ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:18.251137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:18.252137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 322/642] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:18.254270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea74f7c-d038-4892-b822-251889c7d8ba None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:18.268903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:18.300165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ff33c-3f04-4574-8eb3-9a722a49e0ea None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.340611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbca731-b868-45e9-bb36-3c30739b6c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.341587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbca731-b868-45e9-bb36-3c30739b6c45 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.341930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbca731-b868-45e9-bb36-3c30739b6c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.342455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbca731-b868-45e9-bb36-3c30739b6c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.356116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbca731-b868-45e9-bb36-3c30739b6c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:18.357631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbca731-b868-45e9-bb36-3c30739b6c45 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-1bbca731-b868-45e9-bb36-3c30739b6c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:18.372942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:18.374010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:18.375634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:18.375634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:18.375634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:18.375634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 321/643] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:18.375634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbca731-b868-45e9-bb36-3c30739b6c45 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:18.386481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea74f7c-d038-4892-b822-251889c7d8ba None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.430418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87607c37-d620-44c9-b6fb-5b5556855745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.431061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87607c37-d620-44c9-b6fb-5b5556855745 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.431281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87607c37-d620-44c9-b6fb-5b5556855745 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.431571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87607c37-d620-44c9-b6fb-5b5556855745 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.432563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87607c37-d620-44c9-b6fb-5b5556855745 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:18.432864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87607c37-d620-44c9-b6fb-5b5556855745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:18.434277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87607c37-d620-44c9-b6fb-5b5556855745 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:18.443568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 323/644] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:18.444692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:18.448607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87607c37-d620-44c9-b6fb-5b5556855745 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:18.498816 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbca731-b868-45e9-bb36-3c30739b6c45 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.528509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:18.537547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd672965-d543-4b60-bb0b-3743de7d7ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.538190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd672965-d543-4b60-bb0b-3743de7d7ddd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.538471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd672965-d543-4b60-bb0b-3743de7d7ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.538743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd672965-d543-4b60-bb0b-3743de7d7ddd None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.539489 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd672965-d543-4b60-bb0b-3743de7d7ddd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:18.539797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd672965-d543-4b60-bb0b-3743de7d7ddd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:18.541309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd672965-d543-4b60-bb0b-3743de7d7ddd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:18.560864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 322/645] 10.222.0.83 () {56 vars in 1133 bytes} [Wed Aug 24 11:05:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:18.561628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd672965-d543-4b60-bb0b-3743de7d7ddd None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:18.571358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87607c37-d620-44c9-b6fb-5b5556855745 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.610386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db19b7-5765-482d-9cf3-f0ac48bce66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.611032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db19b7-5765-482d-9cf3-f0ac48bce66b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.611556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db19b7-5765-482d-9cf3-f0ac48bce66b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.611556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db19b7-5765-482d-9cf3-f0ac48bce66b None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.612681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db19b7-5765-482d-9cf3-f0ac48bce66b None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:18.612681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db19b7-5765-482d-9cf3-f0ac48bce66b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:18.614044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db19b7-5765-482d-9cf3-f0ac48bce66b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:18.628868 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 324/646] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:18.628868 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db19b7-5765-482d-9cf3-f0ac48bce66b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:18.685311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd672965-d543-4b60-bb0b-3743de7d7ddd None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.700432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:18.722994 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604e6a3-9bed-4f2f-85f4-0caa759e2f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.724032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604e6a3-9bed-4f2f-85f4-0caa759e2f27 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.724584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604e6a3-9bed-4f2f-85f4-0caa759e2f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.724847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604e6a3-9bed-4f2f-85f4-0caa759e2f27 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.725509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604e6a3-9bed-4f2f-85f4-0caa759e2f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:18.725998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604e6a3-9bed-4f2f-85f4-0caa759e2f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:18.727667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604e6a3-9bed-4f2f-85f4-0caa759e2f27 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:18.741047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 323/647] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:18.741907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604e6a3-9bed-4f2f-85f4-0caa759e2f27 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:18.747991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db19b7-5765-482d-9cf3-f0ac48bce66b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.781812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:18.788135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd2079-541d-4649-a0b2-4cc946de173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.789030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd2079-541d-4649-a0b2-4cc946de173a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.789622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd2079-541d-4649-a0b2-4cc946de173a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.790129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd2079-541d-4649-a0b2-4cc946de173a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.802994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd2079-541d-4649-a0b2-4cc946de173a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:18.805026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd2079-541d-4649-a0b2-4cc946de173a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f5cd2079-541d-4649-a0b2-4cc946de173a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:18.821010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:18.822186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:18.823295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:18.823295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:18.823295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:18.828517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 325/648] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:18.829424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd2079-541d-4649-a0b2-4cc946de173a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:18.856932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604e6a3-9bed-4f2f-85f4-0caa759e2f27 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.902278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b405d-f21e-416f-bbf7-64ebe5134705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.902918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b405d-f21e-416f-bbf7-64ebe5134705 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.903200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b405d-f21e-416f-bbf7-64ebe5134705 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.903476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b405d-f21e-416f-bbf7-64ebe5134705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.949646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd2079-541d-4649-a0b2-4cc946de173a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:18.955407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:18.983989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b405d-f21e-416f-bbf7-64ebe5134705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:18.985478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b405d-f21e-416f-bbf7-64ebe5134705 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:18.986499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaaeb33-ea74-45de-ba38-fa33a864eda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:18.987172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaaeb33-ea74-45de-ba38-fa33a864eda9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:18.987488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaaeb33-ea74-45de-ba38-fa33a864eda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:18.987800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaaeb33-ea74-45de-ba38-fa33a864eda9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:18.999372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaaeb33-ea74-45de-ba38-fa33a864eda9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.001584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaaeb33-ea74-45de-ba38-fa33a864eda9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-9eaaeb33-ea74-45de-ba38-fa33a864eda9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:19.018150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:19.019167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:19.020054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:19.020054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:19.020054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:19.020054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 326/649] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:19.020286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaaeb33-ea74-45de-ba38-fa33a864eda9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.059317 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:19.110052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 324/650] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:19.111233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b405d-f21e-416f-bbf7-64ebe5134705 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.132851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaaeb33-ea74-45de-ba38-fa33a864eda9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:19.171485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c229ba-592b-4fc8-adf0-bb988f677f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:19.172241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c229ba-592b-4fc8-adf0-bb988f677f43 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.172522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c229ba-592b-4fc8-adf0-bb988f677f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.172793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c229ba-592b-4fc8-adf0-bb988f677f43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.180424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c229ba-592b-4fc8-adf0-bb988f677f43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.181899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c229ba-592b-4fc8-adf0-bb988f677f43 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.193839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a4c229ba-592b-4fc8-adf0-bb988f677f43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:19.194828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:19.195707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 327/651] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:19.196626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c229ba-592b-4fc8-adf0-bb988f677f43 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.224375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b405d-f21e-416f-bbf7-64ebe5134705 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:19.266754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fb441-974f-4367-81cf-8e13dcdcdc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:19.267528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fb441-974f-4367-81cf-8e13dcdcdc4f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.268150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fb441-974f-4367-81cf-8e13dcdcdc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.268553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fb441-974f-4367-81cf-8e13dcdcdc4f None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.269351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fb441-974f-4367-81cf-8e13dcdcdc4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:19.269652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fb441-974f-4367-81cf-8e13dcdcdc4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.271270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fb441-974f-4367-81cf-8e13dcdcdc4f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.287340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 325/652] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:19.288522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fb441-974f-4367-81cf-8e13dcdcdc4f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.308385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:19.314470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c229ba-592b-4fc8-adf0-bb988f677f43 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:19.353549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925bf8f-105d-41e5-bf98-91729134fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:19.354281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925bf8f-105d-41e5-bf98-91729134fade None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.354564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925bf8f-105d-41e5-bf98-91729134fade None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.354837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925bf8f-105d-41e5-bf98-91729134fade None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.355505 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925bf8f-105d-41e5-bf98-91729134fade None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:19.355811 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925bf8f-105d-41e5-bf98-91729134fade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.357219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925bf8f-105d-41e5-bf98-91729134fade None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.370335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 328/653] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:19.371106 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925bf8f-105d-41e5-bf98-91729134fade None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.422597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fb441-974f-4367-81cf-8e13dcdcdc4f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:19.459944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c569ce8-c334-4ca9-97c9-cf6c05359e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:19.460574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c569ce8-c334-4ca9-97c9-cf6c05359e6b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.460851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c569ce8-c334-4ca9-97c9-cf6c05359e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.461125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c569ce8-c334-4ca9-97c9-cf6c05359e6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.463516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:19.473270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c569ce8-c334-4ca9-97c9-cf6c05359e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.474773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c569ce8-c334-4ca9-97c9-cf6c05359e6b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-7c569ce8-c334-4ca9-97c9-cf6c05359e6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:19.490392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:19.491426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:19.492367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:19.492367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:19.492367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:19.492565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925bf8f-105d-41e5-bf98-91729134fade None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:19.492970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 326/654] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:19.492970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c569ce8-c334-4ca9-97c9-cf6c05359e6b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.531127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a10a-d6f2-431a-be46-40ff8d92fa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:19.532385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a10a-d6f2-431a-be46-40ff8d92fa2b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.532662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a10a-d6f2-431a-be46-40ff8d92fa2b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.532930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a10a-d6f2-431a-be46-40ff8d92fa2b None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.533632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7a10a-d6f2-431a-be46-40ff8d92fa2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:19.533974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7a10a-d6f2-431a-be46-40ff8d92fa2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.535421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7a10a-d6f2-431a-be46-40ff8d92fa2b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.548104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 329/655] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:19.549428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1b6db-97f0-4322-8a31-66503bf7da5a None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.549721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1b6db-97f0-4322-8a31-66503bf7da5a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.550028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1b6db-97f0-4322-8a31-66503bf7da5a None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.551293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 330/656] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:19.553504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1b6db-97f0-4322-8a31-66503bf7da5a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.573383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:19.601234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c569ce8-c334-4ca9-97c9-cf6c05359e6b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:19.638119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68332b-f953-45a7-b6b4-50dfff50f205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:19.638677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68332b-f953-45a7-b6b4-50dfff50f205 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.638945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68332b-f953-45a7-b6b4-50dfff50f205 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.639242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68332b-f953-45a7-b6b4-50dfff50f205 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.648270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68332b-f953-45a7-b6b4-50dfff50f205 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.649733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68332b-f953-45a7-b6b4-50dfff50f205 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.666780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1b6db-97f0-4322-8a31-66503bf7da5a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-3c68332b-f953-45a7-b6b4-50dfff50f205 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:19.667106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:19.668246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 327/657] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:19.669193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68332b-f953-45a7-b6b4-50dfff50f205 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.710711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ce44c-e7e5-4e70-b971-d1807945a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:19.711332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ce44c-e7e5-4e70-b971-d1807945a477 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.711614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ce44c-e7e5-4e70-b971-d1807945a477 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.711888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ce44c-e7e5-4e70-b971-d1807945a477 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.721119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ce44c-e7e5-4e70-b971-d1807945a477 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.722975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ce44c-e7e5-4e70-b971-d1807945a477 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.724379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-690ce44c-e7e5-4e70-b971-d1807945a477 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:19.737512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:19.743458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 331/658] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:19.744688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ce44c-e7e5-4e70-b971-d1807945a477 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.790746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68332b-f953-45a7-b6b4-50dfff50f205 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:19.827646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252dd6-bc75-432c-8afa-6805fedc749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:19.828898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252dd6-bc75-432c-8afa-6805fedc749e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.829176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252dd6-bc75-432c-8afa-6805fedc749e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.829446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252dd6-bc75-432c-8afa-6805fedc749e None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.830131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb252dd6-bc75-432c-8afa-6805fedc749e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:19.830436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb252dd6-bc75-432c-8afa-6805fedc749e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.831906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb252dd6-bc75-432c-8afa-6805fedc749e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.839820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:19.844992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 328/659] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:19.846278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b5b32-8ae5-415e-a525-0ceefced12c5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.846562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b5b32-8ae5-415e-a525-0ceefced12c5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.846833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b5b32-8ae5-415e-a525-0ceefced12c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.865082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-647b5b32-8ae5-415e-a525-0ceefced12c5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:19.866342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-647b5b32-8ae5-415e-a525-0ceefced12c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:19.873006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ce44c-e7e5-4e70-b971-d1807945a477 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:19.920421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef340413-bbc4-4255-83d8-2bf712e24423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:19.920953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef340413-bbc4-4255-83d8-2bf712e24423 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:19.921232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef340413-bbc4-4255-83d8-2bf712e24423 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:19.921506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef340413-bbc4-4255-83d8-2bf712e24423 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:19.922822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef340413-bbc4-4255-83d8-2bf712e24423 None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=f1f214c2b77d4cbab2f15122ba21bf38) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:19.923130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef340413-bbc4-4255-83d8-2bf712e24423 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:19.924584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef340413-bbc4-4255-83d8-2bf712e24423 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:19.930837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 332/660] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/role_assignments?user.id=f1f214c2b77d4cbab2f15122ba21bf38&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:19.931670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef340413-bbc4-4255-83d8-2bf712e24423 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:19.949589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 329/661] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:19.949589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b5b32-8ae5-415e-a525-0ceefced12c5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:20.002367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:20.073216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b5b32-8ae5-415e-a525-0ceefced12c5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:20.114430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef340413-bbc4-4255-83d8-2bf712e24423 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:20.117304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c030f6-3dc3-4059-b9f7-ffb5a480d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:20.117963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c030f6-3dc3-4059-b9f7-ffb5a480d35b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:20.118278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c030f6-3dc3-4059-b9f7-ffb5a480d35b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:20.118555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c030f6-3dc3-4059-b9f7-ffb5a480d35b None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:20.119294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c030f6-3dc3-4059-b9f7-ffb5a480d35b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:20.119595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c030f6-3dc3-4059-b9f7-ffb5a480d35b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:20.122006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c030f6-3dc3-4059-b9f7-ffb5a480d35b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:20.144147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 330/662] 10.222.0.83 () {56 vars in 1139 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:20.145288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c030f6-3dc3-4059-b9f7-ffb5a480d35b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:20.162240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556582e-9418-4439-bef4-f8fb8acb2e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:20.163642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556582e-9418-4439-bef4-f8fb8acb2e6f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:20.163642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556582e-9418-4439-bef4-f8fb8acb2e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:20.163642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556582e-9418-4439-bef4-f8fb8acb2e6f None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:20.164370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556582e-9418-4439-bef4-f8fb8acb2e6f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:20.164752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556582e-9418-4439-bef4-f8fb8acb2e6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:20.166347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556582e-9418-4439-bef4-f8fb8acb2e6f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:20.183560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:20.197368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 333/663] 10.222.0.83 () {56 vars in 1137 bytes} [Wed Aug 24 11:05:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:20.198660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556582e-9418-4439-bef4-f8fb8acb2e6f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:20.268212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:20.276256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c030f6-3dc3-4059-b9f7-ffb5a480d35b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:20.327622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924115e4-9b29-46f5-b3c9-20c1d9761328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:20.328334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924115e4-9b29-46f5-b3c9-20c1d9761328 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:20.328611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924115e4-9b29-46f5-b3c9-20c1d9761328 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:20.328952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924115e4-9b29-46f5-b3c9-20c1d9761328 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:20.331382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556582e-9418-4439-bef4-f8fb8acb2e6f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:20.336385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924115e4-9b29-46f5-b3c9-20c1d9761328 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:20.337846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924115e4-9b29-46f5-b3c9-20c1d9761328 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-924115e4-9b29-46f5-b3c9-20c1d9761328 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:20.350637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:20.351651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 331/664] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:20.352609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924115e4-9b29-46f5-b3c9-20c1d9761328 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:20.375374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54acac3-6acb-4a4e-87ed-876b477d3196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:20.375991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54acac3-6acb-4a4e-87ed-876b477d3196 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:20.376274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54acac3-6acb-4a4e-87ed-876b477d3196 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:20.376551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54acac3-6acb-4a4e-87ed-876b477d3196 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:20.459206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:20.464438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54acac3-6acb-4a4e-87ed-876b477d3196 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:20.467540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54acac3-6acb-4a4e-87ed-876b477d3196 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:20.476123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924115e4-9b29-46f5-b3c9-20c1d9761328 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:20.531003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987cd96-1de6-490b-83d0-15b9221cdaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:20.532588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987cd96-1de6-490b-83d0-15b9221cdaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:20.537495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987cd96-1de6-490b-83d0-15b9221cdaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:20.538051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987cd96-1de6-490b-83d0-15b9221cdaa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:20.565796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:20.619736 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 334/665] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:20.620801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54acac3-6acb-4a4e-87ed-876b477d3196 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:20.622379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987cd96-1de6-490b-83d0-15b9221cdaa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:20.625969 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987cd96-1de6-490b-83d0-15b9221cdaa9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:20.753330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:20.758381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54acac3-6acb-4a4e-87ed-876b477d3196 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:20.776170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 332/666] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:20.776170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987cd96-1de6-490b-83d0-15b9221cdaa9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:20.810076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cab5a-e73c-43bc-9a9c-e2e55a271731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:20.810726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cab5a-e73c-43bc-9a9c-e2e55a271731 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:20.811008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cab5a-e73c-43bc-9a9c-e2e55a271731 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:20.811277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cab5a-e73c-43bc-9a9c-e2e55a271731 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:20.823445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821cab5a-e73c-43bc-9a9c-e2e55a271731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-821cab5a-e73c-43bc-9a9c-e2e55a271731 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:20.828244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:20.829383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:20.830297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 335/667] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:20.830988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cab5a-e73c-43bc-9a9c-e2e55a271731 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:20.860136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:20.908119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987cd96-1de6-490b-83d0-15b9221cdaa9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:20.945028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe046db-97f2-4567-939e-3c6fd1b34252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:20.945633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe046db-97f2-4567-939e-3c6fd1b34252 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:20.945947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe046db-97f2-4567-939e-3c6fd1b34252 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:20.946195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe046db-97f2-4567-939e-3c6fd1b34252 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:20.946910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe046db-97f2-4567-939e-3c6fd1b34252 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:20.947181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe046db-97f2-4567-939e-3c6fd1b34252 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:20.948588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe046db-97f2-4567-939e-3c6fd1b34252 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:20.968422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 333/668] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:20.968422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe046db-97f2-4567-939e-3c6fd1b34252 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:20.973362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cab5a-e73c-43bc-9a9c-e2e55a271731 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.016339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240dcd8-bda7-49a0-8047-b51472698334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.017204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240dcd8-bda7-49a0-8047-b51472698334 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.017327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240dcd8-bda7-49a0-8047-b51472698334 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.017507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240dcd8-bda7-49a0-8047-b51472698334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.028870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240dcd8-bda7-49a0-8047-b51472698334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.030437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240dcd8-bda7-49a0-8047-b51472698334 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d240dcd8-bda7-49a0-8047-b51472698334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:21.045270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:21.046305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:21.047714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:21.047714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:21.047714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:21.047714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 336/669] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:21.047956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240dcd8-bda7-49a0-8047-b51472698334 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.063952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:21.101503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe046db-97f2-4567-939e-3c6fd1b34252 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.117437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:21.137962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a84ac2-c227-4663-befa-29473436ae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.138601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a84ac2-c227-4663-befa-29473436ae2a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.138939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a84ac2-c227-4663-befa-29473436ae2a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.139216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a84ac2-c227-4663-befa-29473436ae2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.150517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a84ac2-c227-4663-befa-29473436ae2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.152069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a84ac2-c227-4663-befa-29473436ae2a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-87a84ac2-c227-4663-befa-29473436ae2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:21.169325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:21.170378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:21.171401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:21.171401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:21.171401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:21.171401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 334/670] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:21.171740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a84ac2-c227-4663-befa-29473436ae2a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.173221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240dcd8-bda7-49a0-8047-b51472698334 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.214707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0ae8e-63dc-483d-bd16-f5b9260dd15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.215374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0ae8e-63dc-483d-bd16-f5b9260dd15a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.215653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0ae8e-63dc-483d-bd16-f5b9260dd15a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.215916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0ae8e-63dc-483d-bd16-f5b9260dd15a None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.216595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0ae8e-63dc-483d-bd16-f5b9260dd15a None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:21.216951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0ae8e-63dc-483d-bd16-f5b9260dd15a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.218495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0ae8e-63dc-483d-bd16-f5b9260dd15a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.233531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 337/671] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:21.233531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0ae8e-63dc-483d-bd16-f5b9260dd15a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.297458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a84ac2-c227-4663-befa-29473436ae2a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.319637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:21.342205 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645770a3-bf6f-4545-bbb2-12f3eaab7d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.342862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645770a3-bf6f-4545-bbb2-12f3eaab7d7a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.343165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645770a3-bf6f-4545-bbb2-12f3eaab7d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.343352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645770a3-bf6f-4545-bbb2-12f3eaab7d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.344625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645770a3-bf6f-4545-bbb2-12f3eaab7d7a None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=32a28f9fb3134f33819e53e0c72c63b4) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:21.344940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645770a3-bf6f-4545-bbb2-12f3eaab7d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.346533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645770a3-bf6f-4545-bbb2-12f3eaab7d7a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.356173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0ae8e-63dc-483d-bd16-f5b9260dd15a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.362340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 335/672] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/role_assignments?user.id=32a28f9fb3134f33819e53e0c72c63b4&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:21.363700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645770a3-bf6f-4545-bbb2-12f3eaab7d7a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.408593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4e4bc-58bf-4702-a0f7-045156223aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.409229 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4e4bc-58bf-4702-a0f7-045156223aeb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.409518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4e4bc-58bf-4702-a0f7-045156223aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.409793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4e4bc-58bf-4702-a0f7-045156223aeb None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.410534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4e4bc-58bf-4702-a0f7-045156223aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:21.411024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4e4bc-58bf-4702-a0f7-045156223aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.411892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:21.412785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4e4bc-58bf-4702-a0f7-045156223aeb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.428947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 338/673] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:21.430216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4e4bc-58bf-4702-a0f7-045156223aeb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.517245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645770a3-bf6f-4545-bbb2-12f3eaab7d7a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.556635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4e4bc-58bf-4702-a0f7-045156223aeb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.568269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2930f-98ca-4643-a106-f5877a107723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.568946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2930f-98ca-4643-a106-f5877a107723 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.569228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2930f-98ca-4643-a106-f5877a107723 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.569634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2930f-98ca-4643-a106-f5877a107723 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.571036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2930f-98ca-4643-a106-f5877a107723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:21.571176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2930f-98ca-4643-a106-f5877a107723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.574442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2930f-98ca-4643-a106-f5877a107723 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.591151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 336/674] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:21.592610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2930f-98ca-4643-a106-f5877a107723 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.605301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:21.611737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7232b1-11b6-45dd-ae79-2f28a1e3148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.612388 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7232b1-11b6-45dd-ae79-2f28a1e3148e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.612671 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7232b1-11b6-45dd-ae79-2f28a1e3148e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.612942 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7232b1-11b6-45dd-ae79-2f28a1e3148e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.627551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7232b1-11b6-45dd-ae79-2f28a1e3148e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.629216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7232b1-11b6-45dd-ae79-2f28a1e3148e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-9c7232b1-11b6-45dd-ae79-2f28a1e3148e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:21.647412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:21.648404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:21.649581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:21.649581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:21.649581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:21.649581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 339/675] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:21.649863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7232b1-11b6-45dd-ae79-2f28a1e3148e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.724483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:21.736055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2930f-98ca-4643-a106-f5877a107723 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.769463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7232b1-11b6-45dd-ae79-2f28a1e3148e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.776652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50e124-fec1-4182-b8a5-ab66c340fe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.777298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50e124-fec1-4182-b8a5-ab66c340fe8a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.777583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50e124-fec1-4182-b8a5-ab66c340fe8a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.777860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50e124-fec1-4182-b8a5-ab66c340fe8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.786381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50e124-fec1-4182-b8a5-ab66c340fe8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.787975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50e124-fec1-4182-b8a5-ab66c340fe8a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-cb50e124-fec1-4182-b8a5-ab66c340fe8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:21.801818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:21.802956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:21.804128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:21.804128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:21.804128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:21.804128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:21.804128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:21.804128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:21.804128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:21.804128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:21.804128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 337/676] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:21.804624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50e124-fec1-4182-b8a5-ab66c340fe8a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.812896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d2499-6758-47c2-8960-585f15b57fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.813757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d2499-6758-47c2-8960-585f15b57fde None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.814217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d2499-6758-47c2-8960-585f15b57fde None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.814695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d2499-6758-47c2-8960-585f15b57fde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.822869 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d2499-6758-47c2-8960-585f15b57fde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.824648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d2499-6758-47c2-8960-585f15b57fde None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f18d2499-6758-47c2-8960-585f15b57fde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:21.837611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:21.838823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:21.839901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:21.839901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:21.839901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:21.839901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:21.839901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:21.839901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:21.839901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:21.839901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:21.846501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 340/677] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:21.847718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d2499-6758-47c2-8960-585f15b57fde None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.897753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:21.925758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50e124-fec1-4182-b8a5-ab66c340fe8a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.958357 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d2499-6758-47c2-8960-585f15b57fde None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:21.968136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f5a0c-7e1f-416b-afb5-c2a4e0da3d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.969058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f5a0c-7e1f-416b-afb5-c2a4e0da3d85 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.969180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f5a0c-7e1f-416b-afb5-c2a4e0da3d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.969429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f5a0c-7e1f-416b-afb5-c2a4e0da3d85 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:21.970119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f5a0c-7e1f-416b-afb5-c2a4e0da3d85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:21.970426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f5a0c-7e1f-416b-afb5-c2a4e0da3d85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:21.971985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f5a0c-7e1f-416b-afb5-c2a4e0da3d85 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:21.991459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 338/678] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:21.992512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f5a0c-7e1f-416b-afb5-c2a4e0da3d85 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:21.998552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c3b93-95a9-493f-825b-67b0dff6f141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:21.999079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c3b93-95a9-493f-825b-67b0dff6f141 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:21.999366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c3b93-95a9-493f-825b-67b0dff6f141 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:21.999647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c3b93-95a9-493f-825b-67b0dff6f141 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:22.001004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c3b93-95a9-493f-825b-67b0dff6f141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:22.001400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c3b93-95a9-493f-825b-67b0dff6f141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:22.003105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c3b93-95a9-493f-825b-67b0dff6f141 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:22.011197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:22.018562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 341/679] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:22.019865 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c3b93-95a9-493f-825b-67b0dff6f141 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:22.108200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f5a0c-7e1f-416b-afb5-c2a4e0da3d85 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:22.142851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c3b93-95a9-493f-825b-67b0dff6f141 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:22.145466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:22.148583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b092b2b-afa1-4477-8769-967317b302b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:22.149191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b092b2b-afa1-4477-8769-967317b302b5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:22.149471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b092b2b-afa1-4477-8769-967317b302b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:22.149744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b092b2b-afa1-4477-8769-967317b302b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:22.150478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b092b2b-afa1-4477-8769-967317b302b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:22.150786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b092b2b-afa1-4477-8769-967317b302b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:22.152361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b092b2b-afa1-4477-8769-967317b302b5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:22.166028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 339/680] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:22.166887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b092b2b-afa1-4477-8769-967317b302b5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:22.191925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4b2df-2dc5-4d19-9f51-9c4d9f7ebb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:22.193044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4b2df-2dc5-4d19-9f51-9c4d9f7ebb8b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:22.193661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4b2df-2dc5-4d19-9f51-9c4d9f7ebb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:22.194159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4b2df-2dc5-4d19-9f51-9c4d9f7ebb8b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:22.202201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4b2df-2dc5-4d19-9f51-9c4d9f7ebb8b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:22.203767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4b2df-2dc5-4d19-9f51-9c4d9f7ebb8b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-2cf4b2df-2dc5-4d19-9f51-9c4d9f7ebb8b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:22.216287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:22.217386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:22.218380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 342/681] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:22.219499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93740-29a2-461b-9f8b-8f7ced10abe6 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:22.219499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93740-29a2-461b-9f8b-8f7ced10abe6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:22.219499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93740-29a2-461b-9f8b-8f7ced10abe6 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:22.222227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 343/682] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:22.223284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93740-29a2-461b-9f8b-8f7ced10abe6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:22.256650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:22.291735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b092b2b-afa1-4477-8769-967317b302b5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:22.331405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800850dd-b202-4421-8e00-bfff3b5c9073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:22.332100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800850dd-b202-4421-8e00-bfff3b5c9073 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:22.332382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800850dd-b202-4421-8e00-bfff3b5c9073 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:22.332711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800850dd-b202-4421-8e00-bfff3b5c9073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:22.333383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800850dd-b202-4421-8e00-bfff3b5c9073 None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=b0145242ef784168bada8348ccd95f22) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:22.333737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800850dd-b202-4421-8e00-bfff3b5c9073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:22.335434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800850dd-b202-4421-8e00-bfff3b5c9073 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:22.349903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 340/683] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:22 2022] GET /identity/v3/role_assignments?user.id=b0145242ef784168bada8348ccd95f22&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:22.349903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800850dd-b202-4421-8e00-bfff3b5c9073 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:22.354488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93740-29a2-461b-9f8b-8f7ced10abe6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:22.400808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb913fee-5d2c-4260-822b-7adccbae84d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:22.401699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb913fee-5d2c-4260-822b-7adccbae84d2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:22.402124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb913fee-5d2c-4260-822b-7adccbae84d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:22.402476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb913fee-5d2c-4260-822b-7adccbae84d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:22.403348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb913fee-5d2c-4260-822b-7adccbae84d2 None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=f9dd6f187dd84f04866da64732b242d7) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:22.403662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb913fee-5d2c-4260-822b-7adccbae84d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:22.405171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb913fee-5d2c-4260-822b-7adccbae84d2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:22.412295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 344/684] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:22 2022] GET /identity/v3/role_assignments?user.id=f9dd6f187dd84f04866da64732b242d7&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:22.419925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ec7f1-dda1-4070-8d20-9d8ff6b90bf1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:22.419925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ec7f1-dda1-4070-8d20-9d8ff6b90bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:22.420190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ec7f1-dda1-4070-8d20-9d8ff6b90bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:22.478953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fa9ec7f1-dda1-4070-8d20-9d8ff6b90bf1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:22.480239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fa9ec7f1-dda1-4070-8d20-9d8ff6b90bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:22.510775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800850dd-b202-4421-8e00-bfff3b5c9073 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:22.547420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:22.560475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691786a-7bca-4086-9abc-4b112f74c033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:22.563492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691786a-7bca-4086-9abc-4b112f74c033 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:22.563492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691786a-7bca-4086-9abc-4b112f74c033 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:22.563492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691786a-7bca-4086-9abc-4b112f74c033 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:22.565495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691786a-7bca-4086-9abc-4b112f74c033 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:22.565990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691786a-7bca-4086-9abc-4b112f74c033 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:22.567689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691786a-7bca-4086-9abc-4b112f74c033 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:22.569742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:22.592812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 341/685] 10.222.0.83 () {56 vars in 1135 bytes} [Wed Aug 24 11:05:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:22.599609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691786a-7bca-4086-9abc-4b112f74c033 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:22.613938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 345/686] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:22.728641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691786a-7bca-4086-9abc-4b112f74c033 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:22.771299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7c078-dac7-4deb-80e5-2f9198702086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:22.771917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7c078-dac7-4deb-80e5-2f9198702086 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:22.772196 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7c078-dac7-4deb-80e5-2f9198702086 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:22.772471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7c078-dac7-4deb-80e5-2f9198702086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:22.838645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:22.852535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7c078-dac7-4deb-80e5-2f9198702086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:22.854033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7c078-dac7-4deb-80e5-2f9198702086 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:22.913729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ec7f1-dda1-4070-8d20-9d8ff6b90bf1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:23.012375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 342/687] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:23.028313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ec7f1-dda1-4070-8d20-9d8ff6b90bf1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:23.032521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7c078-dac7-4deb-80e5-2f9198702086 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:23.074436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f74a1-5bfe-4219-ae21-92a85d004205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:23.075081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f74a1-5bfe-4219-ae21-92a85d004205 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:23.075484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f74a1-5bfe-4219-ae21-92a85d004205 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:23.075766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f74a1-5bfe-4219-ae21-92a85d004205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:23.088129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f74a1-5bfe-4219-ae21-92a85d004205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:23.089631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f74a1-5bfe-4219-ae21-92a85d004205 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-183f74a1-5bfe-4219-ae21-92a85d004205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:23.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:23.108434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:23.109588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:23.109588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:23.109588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:23.109588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 346/688] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:23.114584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f74a1-5bfe-4219-ae21-92a85d004205 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:23.128984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:23.226518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f74a1-5bfe-4219-ae21-92a85d004205 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:23.263969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a95a5-c6f8-4d15-8657-1595226271a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:23.264507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a95a5-c6f8-4d15-8657-1595226271a2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:23.264792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a95a5-c6f8-4d15-8657-1595226271a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:23.265068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a95a5-c6f8-4d15-8657-1595226271a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:23.265577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:23.265790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a95a5-c6f8-4d15-8657-1595226271a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:23.266168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a95a5-c6f8-4d15-8657-1595226271a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:23.268378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a95a5-c6f8-4d15-8657-1595226271a2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:23.272509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7c078-dac7-4deb-80e5-2f9198702086 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:23.280890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 347/689] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:23.291396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a95a5-c6f8-4d15-8657-1595226271a2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:23.309373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a527c-0836-4b98-beea-65690719aa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:23.310053 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a527c-0836-4b98-beea-65690719aa53 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:23.310333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a527c-0836-4b98-beea-65690719aa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:23.311026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a527c-0836-4b98-beea-65690719aa53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:23.325304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a527c-0836-4b98-beea-65690719aa53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:23.327346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a527c-0836-4b98-beea-65690719aa53 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c04a527c-0836-4b98-beea-65690719aa53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:23.345539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:23.346678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:23.347592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:23.347592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:23.347592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:23.349625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 343/690] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:23.356136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a527c-0836-4b98-beea-65690719aa53 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:23.487844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a527c-0836-4b98-beea-65690719aa53 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:23.519380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:23.531940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1581e-ba9d-4158-91a9-7b2f46b4a68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:23.534453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1581e-ba9d-4158-91a9-7b2f46b4a68e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:23.534453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1581e-ba9d-4158-91a9-7b2f46b4a68e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:23.534453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1581e-ba9d-4158-91a9-7b2f46b4a68e None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:23.534453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1581e-ba9d-4158-91a9-7b2f46b4a68e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:23.534453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1581e-ba9d-4158-91a9-7b2f46b4a68e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:23.535534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1581e-ba9d-4158-91a9-7b2f46b4a68e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:23.552420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 344/691] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:23.557923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1581e-ba9d-4158-91a9-7b2f46b4a68e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:23.575621 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:23.590226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a95a5-c6f8-4d15-8657-1595226271a2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:23.659516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2e198-aacd-463b-9786-009d61308ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:23.660141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2e198-aacd-463b-9786-009d61308ead None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:23.660421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2e198-aacd-463b-9786-009d61308ead None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:23.660694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2e198-aacd-463b-9786-009d61308ead None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:23.661365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2e198-aacd-463b-9786-009d61308ead None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:23.662960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2e198-aacd-463b-9786-009d61308ead None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:23.725732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1581e-ba9d-4158-91a9-7b2f46b4a68e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:23.763984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ef03f-171b-4d7f-905b-61c440df1634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:23.764620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ef03f-171b-4d7f-905b-61c440df1634 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:23.764904 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ef03f-171b-4d7f-905b-61c440df1634 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:23.765180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ef03f-171b-4d7f-905b-61c440df1634 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:23.772766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ef03f-171b-4d7f-905b-61c440df1634 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:23.774708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ef03f-171b-4d7f-905b-61c440df1634 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:23.779477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 348/692] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:05:23 2022] POST /identity/v3/users => generated 254 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:23.780727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2f379-b7a1-4dc4-8071-0e17879e1713 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:23.781035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2f379-b7a1-4dc4-8071-0e17879e1713 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:23.781343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2f379-b7a1-4dc4-8071-0e17879e1713 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:23.787659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 349/693] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:23.789119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6b77-03f2-425a-bbb2-9ab699634954 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:23.789406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6b77-03f2-425a-bbb2-9ab699634954 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:23.789689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6b77-03f2-425a-bbb2-9ab699634954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-af4ef03f-171b-4d7f-905b-61c440df1634 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:23.797209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:23.804265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:23.805161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:23.805161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:23.805161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:23.805161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:23.805161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:23.805161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:23.805161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:23.805161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:23.805161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 345/694] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:23.814953 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ef03f-171b-4d7f-905b-61c440df1634 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:23.816900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-80ca6b77-03f2-425a-bbb2-9ab699634954 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:23.818575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-80ca6b77-03f2-425a-bbb2-9ab699634954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:23.911192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 350/695] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:05:23.951126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:23.994354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ef03f-171b-4d7f-905b-61c440df1634 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:24.034784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b6770-c1c4-46ca-a697-89f6711b0516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:24.035420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b6770-c1c4-46ca-a697-89f6711b0516 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:24.035758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b6770-c1c4-46ca-a697-89f6711b0516 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:24.036040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b6770-c1c4-46ca-a697-89f6711b0516 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:24.036757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b6770-c1c4-46ca-a697-89f6711b0516 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:24.037064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b6770-c1c4-46ca-a697-89f6711b0516 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:24.038556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b6770-c1c4-46ca-a697-89f6711b0516 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:24.052962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 346/696] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:24.059646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca6b77-03f2-425a-bbb2-9ab699634954 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:24.128459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:24.187004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca6b77-03f2-425a-bbb2-9ab699634954 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:24.228091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c151fb-d2c4-414e-a4d3-73fa765cbcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:24.229109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c151fb-d2c4-414e-a4d3-73fa765cbcbf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:24.229597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c151fb-d2c4-414e-a4d3-73fa765cbcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:24.230105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c151fb-d2c4-414e-a4d3-73fa765cbcbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:24.231050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c151fb-d2c4-414e-a4d3-73fa765cbcbf None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=12eafcbf5c98425782138d5363db9a25) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:24.231569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c151fb-d2c4-414e-a4d3-73fa765cbcbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:24.233236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c151fb-d2c4-414e-a4d3-73fa765cbcbf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:24.240528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 351/697] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:24 2022] GET /identity/v3/role_assignments?user.id=12eafcbf5c98425782138d5363db9a25&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:24.348887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b6770-c1c4-46ca-a697-89f6711b0516 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:24.467357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b6770-c1c4-46ca-a697-89f6711b0516 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:24.500612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:24.515088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927d022-a523-4251-9e16-5ee70d0848bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:24.516658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927d022-a523-4251-9e16-5ee70d0848bb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:24.517106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927d022-a523-4251-9e16-5ee70d0848bb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:24.517579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927d022-a523-4251-9e16-5ee70d0848bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:24.530397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927d022-a523-4251-9e16-5ee70d0848bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4927d022-a523-4251-9e16-5ee70d0848bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:24.536822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:24.537782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:24.539309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:24.541698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 347/698] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:24.547979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c151fb-d2c4-414e-a4d3-73fa765cbcbf None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:24.655939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473f785-ba60-40a3-918a-48efe7c4159d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:24.657175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473f785-ba60-40a3-918a-48efe7c4159d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:24.657510 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473f785-ba60-40a3-918a-48efe7c4159d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:24.659425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 348/699] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:05:24.665848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ac79f-a466-4645-813a-46c7c27cbf64 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:24.666645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ac79f-a466-4645-813a-46c7c27cbf64 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:24.667146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ac79f-a466-4645-813a-46c7c27cbf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:24.677110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c151fb-d2c4-414e-a4d3-73fa765cbcbf None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:24.691569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-030ac79f-a466-4645-813a-46c7c27cbf64 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:24.693131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-030ac79f-a466-4645-813a-46c7c27cbf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:24.702442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:24.723468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894f59f-624d-4a88-81c3-36d18a6c653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:24.724088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894f59f-624d-4a88-81c3-36d18a6c653f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:24.724372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894f59f-624d-4a88-81c3-36d18a6c653f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:24.724648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894f59f-624d-4a88-81c3-36d18a6c653f None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:24.725630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894f59f-624d-4a88-81c3-36d18a6c653f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:24.725971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894f59f-624d-4a88-81c3-36d18a6c653f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:24.728167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894f59f-624d-4a88-81c3-36d18a6c653f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:24.741961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 352/700] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:24.749615 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894f59f-624d-4a88-81c3-36d18a6c653f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:24.781020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 349/701] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:05:24.871720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894f59f-624d-4a88-81c3-36d18a6c653f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:24.910489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c8677-2638-4614-98e6-e8e94fd7eccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:24.911059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c8677-2638-4614-98e6-e8e94fd7eccf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:24.911342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c8677-2638-4614-98e6-e8e94fd7eccf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:24.911617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c8677-2638-4614-98e6-e8e94fd7eccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:24.924739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c8677-2638-4614-98e6-e8e94fd7eccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:24.925072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c8677-2638-4614-98e6-e8e94fd7eccf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a28c8677-2638-4614-98e6-e8e94fd7eccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:24.942046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:24.943089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:24.943976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:24.943976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:24.943976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:24.943976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 353/702] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:24.949344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ac79f-a466-4645-813a-46c7c27cbf64 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:25.104757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ac79f-a466-4645-813a-46c7c27cbf64 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:25.125961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:25.156024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68d257-a4f3-4dbc-b589-394757bac1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:25.156659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d257-a4f3-4dbc-b589-394757bac1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:25.156993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d257-a4f3-4dbc-b589-394757bac1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:25.157273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d257-a4f3-4dbc-b589-394757bac1a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:25.158001 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68d257-a4f3-4dbc-b589-394757bac1a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:25.158308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68d257-a4f3-4dbc-b589-394757bac1a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:25.159704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68d257-a4f3-4dbc-b589-394757bac1a8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:25.167260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657512e7-2ce6-425d-85cb-f0fa66a68194 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:25.167734 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657512e7-2ce6-425d-85cb-f0fa66a68194 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:25.167950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657512e7-2ce6-425d-85cb-f0fa66a68194 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:25.170350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 354/703] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:05:25.176631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00fe67-2b6f-44d3-bc6c-00db0b65ea60 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:25.177700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00fe67-2b6f-44d3-bc6c-00db0b65ea60 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:25.178123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00fe67-2b6f-44d3-bc6c-00db0b65ea60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:25.183745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 350/704] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:25.184976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68d257-a4f3-4dbc-b589-394757bac1a8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:25.201763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ce00fe67-2b6f-44d3-bc6c-00db0b65ea60 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:25.203196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ce00fe67-2b6f-44d3-bc6c-00db0b65ea60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:25.238467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:25.297472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 355/705] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:05:25.300334 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00fe67-2b6f-44d3-bc6c-00db0b65ea60 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:25.345029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68d257-a4f3-4dbc-b589-394757bac1a8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:25.439347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00fe67-2b6f-44d3-bc6c-00db0b65ea60 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:25.451510 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaba9ca-80ea-4a1a-96e0-b6793cb54a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:25.452158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba9ca-80ea-4a1a-96e0-b6793cb54a98 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:25.452435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba9ca-80ea-4a1a-96e0-b6793cb54a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:25.452705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba9ca-80ea-4a1a-96e0-b6793cb54a98 None admin] PATH_INFO: `/v3/services/network` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:25.453345 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaba9ca-80ea-4a1a-96e0-b6793cb54a98 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:25.454747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaba9ca-80ea-4a1a-96e0-b6793cb54a98 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-aaaba9ca-80ea-4a1a-96e0-b6793cb54a98 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:25.484042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:25.485659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:25.486597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 351/706] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:25 2022] GET /identity/v3/services/network => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:25.493514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d8fcc-6eed-461c-b85f-d3a2dec4ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:25.494287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d8fcc-6eed-461c-b85f-d3a2dec4ba55 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:25.494566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d8fcc-6eed-461c-b85f-d3a2dec4ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:25.494835 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d8fcc-6eed-461c-b85f-d3a2dec4ba55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:25.495509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaba9ca-80ea-4a1a-96e0-b6793cb54a98 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:25.511168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d8fcc-6eed-461c-b85f-d3a2dec4ba55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:25.513017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d8fcc-6eed-461c-b85f-d3a2dec4ba55 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3a6d8fcc-6eed-461c-b85f-d3a2dec4ba55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:25.533182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:25.534224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:25.535145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:25.535678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 356/707] 10.222.0.83 () {56 vars in 1093 bytes} [Wed Aug 24 11:05:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:25.541604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d8fcc-6eed-461c-b85f-d3a2dec4ba55 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:25.573784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:25.581360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:25.664814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaba9ca-80ea-4a1a-96e0-b6793cb54a98 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:25.687105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d8fcc-6eed-461c-b85f-d3a2dec4ba55 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:25.717428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964ca41-01ff-4663-ae4c-00fae95f5665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:25.718078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964ca41-01ff-4663-ae4c-00fae95f5665 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:25.718394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964ca41-01ff-4663-ae4c-00fae95f5665 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:25.719252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964ca41-01ff-4663-ae4c-00fae95f5665 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:25.719918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964ca41-01ff-4663-ae4c-00fae95f5665 None admin] RBAC: Adding query filter params (name=network) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:25.720253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964ca41-01ff-4663-ae4c-00fae95f5665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:25.721776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964ca41-01ff-4663-ae4c-00fae95f5665 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:25.730230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 352/708] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:05:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:25.731104 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964ca41-01ff-4663-ae4c-00fae95f5665 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:25.731333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af226e-8750-4414-8043-25be26129b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:25.732181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af226e-8750-4414-8043-25be26129b44 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:25.732687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af226e-8750-4414-8043-25be26129b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:25.733162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af226e-8750-4414-8043-25be26129b44 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:25.734180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af226e-8750-4414-8043-25be26129b44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:25.735199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af226e-8750-4414-8043-25be26129b44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:25.737082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af226e-8750-4414-8043-25be26129b44 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:25.757703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 357/709] 10.222.0.83 () {56 vars in 1133 bytes} [Wed Aug 24 11:05:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:25.758993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af226e-8750-4414-8043-25be26129b44 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:25.848256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:25.851726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964ca41-01ff-4663-ae4c-00fae95f5665 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:25.868241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af226e-8750-4414-8043-25be26129b44 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:25.878320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:25.891337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9caf8-ae26-4051-93a0-f64958f1145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:25.892395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9caf8-ae26-4051-93a0-f64958f1145d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:25.892928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9caf8-ae26-4051-93a0-f64958f1145d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:25.893457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9caf8-ae26-4051-93a0-f64958f1145d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:25.894644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9caf8-ae26-4051-93a0-f64958f1145d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:25.902331 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9caf8-ae26-4051-93a0-f64958f1145d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4fc9caf8-ae26-4051-93a0-f64958f1145d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:25.908218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:25.909330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:05:25.910227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:25.910993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90280cb3-5c94-46ac-8d5d-2caef4908b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:25.910993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90280cb3-5c94-46ac-8d5d-2caef4908b38 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:25.910993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90280cb3-5c94-46ac-8d5d-2caef4908b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:25.911233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90280cb3-5c94-46ac-8d5d-2caef4908b38 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:25.911913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90280cb3-5c94-46ac-8d5d-2caef4908b38 None admin] RBAC: Adding query filter params (type=network) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:25.912405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90280cb3-5c94-46ac-8d5d-2caef4908b38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:25.918460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90280cb3-5c94-46ac-8d5d-2caef4908b38 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:25.918961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 353/710] 10.222.0.83 () {56 vars in 1129 bytes} [Wed Aug 24 11:05:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:25.920166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9caf8-ae26-4051-93a0-f64958f1145d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:25.924270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 358/711] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:05:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:25.924270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90280cb3-5c94-46ac-8d5d-2caef4908b38 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:26.038090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90280cb3-5c94-46ac-8d5d-2caef4908b38 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:26.079563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54cc93-8373-4181-aec3-803ff73189bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:26.080196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54cc93-8373-4181-aec3-803ff73189bd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:26.080483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54cc93-8373-4181-aec3-803ff73189bd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:26.080761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54cc93-8373-4181-aec3-803ff73189bd None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:26.081416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54cc93-8373-4181-aec3-803ff73189bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:26.081732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54cc93-8373-4181-aec3-803ff73189bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:26.083219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54cc93-8373-4181-aec3-803ff73189bd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:26.088763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 359/712] 10.222.0.83 () {56 vars in 1124 bytes} [Wed Aug 24 11:05:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:26.095172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54cc93-8373-4181-aec3-803ff73189bd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:26.120760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:26.157227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9caf8-ae26-4051-93a0-f64958f1145d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:26.197807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92c926-b4ac-4647-9492-9f4ee3b4ae99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:26.198482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92c926-b4ac-4647-9492-9f4ee3b4ae99 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:26.198800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92c926-b4ac-4647-9492-9f4ee3b4ae99 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:26.199113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92c926-b4ac-4647-9492-9f4ee3b4ae99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:26.213645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54cc93-8373-4181-aec3-803ff73189bd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:26.241677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:26.251744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e6a0b-9570-4207-8549-59c0e8a7808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:26.252411 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e6a0b-9570-4207-8549-59c0e8a7808d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:26.252747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e6a0b-9570-4207-8549-59c0e8a7808d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:26.253026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e6a0b-9570-4207-8549-59c0e8a7808d None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:26.253682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e6a0b-9570-4207-8549-59c0e8a7808d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:26.254057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e6a0b-9570-4207-8549-59c0e8a7808d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:26.255454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e6a0b-9570-4207-8549-59c0e8a7808d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:26.260926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 360/713] 10.222.0.83 () {56 vars in 1124 bytes} [Wed Aug 24 11:05:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:26.282886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92c926-b4ac-4647-9492-9f4ee3b4ae99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:26.284401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92c926-b4ac-4647-9492-9f4ee3b4ae99 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:26.405907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 354/714] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:26.413441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e6a0b-9570-4207-8549-59c0e8a7808d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:26.488115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e30c8-db98-42af-b52e-eef45b8849f8 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:26.488464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e30c8-db98-42af-b52e-eef45b8849f8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:26.488747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e30c8-db98-42af-b52e-eef45b8849f8 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:26.490483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 355/715] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:26.498304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f781f-e6ab-4668-b02b-391f82aee24e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:26.498594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f781f-e6ab-4668-b02b-391f82aee24e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:26.499374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f781f-e6ab-4668-b02b-391f82aee24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:26.530821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b39f781f-e6ab-4668-b02b-391f82aee24e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:26.532751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b39f781f-e6ab-4668-b02b-391f82aee24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:26.540775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:26.569165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e6a0b-9570-4207-8549-59c0e8a7808d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:26.606890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:26.608126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b72a-196d-429b-9c70-93c21cdceb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:26.609004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b72a-196d-429b-9c70-93c21cdceb8d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:26.610057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b72a-196d-429b-9c70-93c21cdceb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:26.610545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b72a-196d-429b-9c70-93c21cdceb8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:26.627244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 356/716] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:05:26.632857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6b72a-196d-429b-9c70-93c21cdceb8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:26.634664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6b72a-196d-429b-9c70-93c21cdceb8d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-91c6b72a-196d-429b-9c70-93c21cdceb8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:26.650371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:26.651604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:26.652699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:26.652699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:26.652699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:26.654542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 361/717] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:26.662250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f781f-e6ab-4668-b02b-391f82aee24e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:26.806849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f781f-e6ab-4668-b02b-391f82aee24e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:26.861222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517727b9-f6e0-4e5c-8306-c37f7306fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:26.861761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517727b9-f6e0-4e5c-8306-c37f7306fd96 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:26.862073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517727b9-f6e0-4e5c-8306-c37f7306fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:26.862347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517727b9-f6e0-4e5c-8306-c37f7306fd96 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:26.863005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517727b9-f6e0-4e5c-8306-c37f7306fd96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:26.863303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517727b9-f6e0-4e5c-8306-c37f7306fd96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:26.864675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517727b9-f6e0-4e5c-8306-c37f7306fd96 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:26.878529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 357/718] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:26.884055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b72a-196d-429b-9c70-93c21cdceb8d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:26.906159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16ade-25fd-4a6c-ad46-ba46ff1517c4 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:26.906471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16ade-25fd-4a6c-ad46-ba46ff1517c4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:26.906715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16ade-25fd-4a6c-ad46-ba46ff1517c4 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:26.908138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 358/719] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:05:26.913441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca7bfe-d310-4513-8e64-441e646c4d30 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:26.913718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca7bfe-d310-4513-8e64-441e646c4d30 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:26.914032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca7bfe-d310-4513-8e64-441e646c4d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:26.942439 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f2ca7bfe-d310-4513-8e64-441e646c4d30 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:26.943647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f2ca7bfe-d310-4513-8e64-441e646c4d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:27.043241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:27.067560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b72a-196d-429b-9c70-93c21cdceb8d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:27.068787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 359/720] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:05:27.070126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca7bfe-d310-4513-8e64-441e646c4d30 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:27.139997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:27.155561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ebc4c-64e2-4a2f-b031-58f3f04d0483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:27.157743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ebc4c-64e2-4a2f-b031-58f3f04d0483 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:27.161951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ebc4c-64e2-4a2f-b031-58f3f04d0483 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:27.162717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ebc4c-64e2-4a2f-b031-58f3f04d0483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:27.175807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ebc4c-64e2-4a2f-b031-58f3f04d0483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:27.177312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ebc4c-64e2-4a2f-b031-58f3f04d0483 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-4a8ebc4c-64e2-4a2f-b031-58f3f04d0483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:27.203470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:27.204761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:27.208805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:27.208805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:27.208805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:27.208805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:27.208805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:27.208805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:27.208805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:27.208805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:27.210389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 362/721] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:27.217910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ebc4c-64e2-4a2f-b031-58f3f04d0483 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:27.232871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca7bfe-d310-4513-8e64-441e646c4d30 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:27.279070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018487a3-badf-4744-94a6-bcb49905de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:27.279732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018487a3-badf-4744-94a6-bcb49905de20 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:27.279995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018487a3-badf-4744-94a6-bcb49905de20 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:27.280269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018487a3-badf-4744-94a6-bcb49905de20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:27.295457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018487a3-badf-4744-94a6-bcb49905de20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-018487a3-badf-4744-94a6-bcb49905de20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:27.300974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:27.301961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:27.302796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 360/722] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:27.310837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018487a3-badf-4744-94a6-bcb49905de20 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:27.389075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ebc4c-64e2-4a2f-b031-58f3f04d0483 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:27.400327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:27.451895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2452bf7-a792-4e1b-be29-9e07db626b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:27.452519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2452bf7-a792-4e1b-be29-9e07db626b46 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:27.452825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2452bf7-a792-4e1b-be29-9e07db626b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:27.453733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2452bf7-a792-4e1b-be29-9e07db626b46 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:27.455641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2452bf7-a792-4e1b-be29-9e07db626b46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:27.455959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2452bf7-a792-4e1b-be29-9e07db626b46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:27.457484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2452bf7-a792-4e1b-be29-9e07db626b46 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:27.473393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 363/723] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:27.474329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2452bf7-a792-4e1b-be29-9e07db626b46 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:27.483650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018487a3-badf-4744-94a6-bcb49905de20 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:27.525924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b87e35-836e-43a8-939b-a65812ebea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:27.526967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b87e35-836e-43a8-939b-a65812ebea4f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:27.527481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b87e35-836e-43a8-939b-a65812ebea4f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:27.527956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b87e35-836e-43a8-939b-a65812ebea4f None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:27.528838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b87e35-836e-43a8-939b-a65812ebea4f None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:27.529351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b87e35-836e-43a8-939b-a65812ebea4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:27.531087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b87e35-836e-43a8-939b-a65812ebea4f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:27.534257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:27.541248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 361/724] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:27.542481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b87e35-836e-43a8-939b-a65812ebea4f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:27.593407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2452bf7-a792-4e1b-be29-9e07db626b46 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:27.633746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f31fe-4d77-4eb2-a28c-24b4bba7281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:27.634576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f31fe-4d77-4eb2-a28c-24b4bba7281f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:27.634759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f31fe-4d77-4eb2-a28c-24b4bba7281f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:27.635028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f31fe-4d77-4eb2-a28c-24b4bba7281f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:27.647273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f31fe-4d77-4eb2-a28c-24b4bba7281f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-218f31fe-4d77-4eb2-a28c-24b4bba7281f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:27.652729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:27.653659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:27.654619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:27.659160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 364/725] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:27.660083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f31fe-4d77-4eb2-a28c-24b4bba7281f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:27.662260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b87e35-836e-43a8-939b-a65812ebea4f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:27.672844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:27.700871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b74ab1-52d4-4094-8305-a75c0d93fdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:27.702187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b74ab1-52d4-4094-8305-a75c0d93fdbd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:27.702187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b74ab1-52d4-4094-8305-a75c0d93fdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:27.702187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b74ab1-52d4-4094-8305-a75c0d93fdbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:27.703209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b74ab1-52d4-4094-8305-a75c0d93fdbd None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=32a28f9fb3134f33819e53e0c72c63b4) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:27.703616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b74ab1-52d4-4094-8305-a75c0d93fdbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:27.705163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b74ab1-52d4-4094-8305-a75c0d93fdbd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:27.713261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 362/726] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:27 2022] GET /identity/v3/role_assignments?user.id=32a28f9fb3134f33819e53e0c72c63b4&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:27.722448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b74ab1-52d4-4094-8305-a75c0d93fdbd None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:27.786283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f31fe-4d77-4eb2-a28c-24b4bba7281f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:27.838321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:27.847519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7f48e-f973-469c-9ff4-17af38e3471b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:27.848183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7f48e-f973-469c-9ff4-17af38e3471b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:27.848465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7f48e-f973-469c-9ff4-17af38e3471b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:27.849182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7f48e-f973-469c-9ff4-17af38e3471b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:27.874144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7f48e-f973-469c-9ff4-17af38e3471b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:27.877591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7f48e-f973-469c-9ff4-17af38e3471b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-4ce7f48e-f973-469c-9ff4-17af38e3471b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:27.905521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:27.906623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:27.907536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:27.907536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:27.907536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:27.907536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 365/727] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:27.914756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7f48e-f973-469c-9ff4-17af38e3471b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:27.924076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b74ab1-52d4-4094-8305-a75c0d93fdbd None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:28.003131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9f79-69f6-47fc-bad3-eb963d946c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:28.003961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9f79-69f6-47fc-bad3-eb963d946c54 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.004245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9f79-69f6-47fc-bad3-eb963d946c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.004523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9f79-69f6-47fc-bad3-eb963d946c54 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.005209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a9f79-69f6-47fc-bad3-eb963d946c54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:28.005514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a9f79-69f6-47fc-bad3-eb963d946c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:28.006944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a9f79-69f6-47fc-bad3-eb963d946c54 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:28.029675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 363/728] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:28.052049 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9f79-69f6-47fc-bad3-eb963d946c54 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:28.122814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:28.131346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7f48e-f973-469c-9ff4-17af38e3471b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:28.185976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111a29d-c999-4c43-82d0-9c144559a51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:28.185976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111a29d-c999-4c43-82d0-9c144559a51a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.185976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111a29d-c999-4c43-82d0-9c144559a51a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.185976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111a29d-c999-4c43-82d0-9c144559a51a None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.186386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111a29d-c999-4c43-82d0-9c144559a51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:28.186386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111a29d-c999-4c43-82d0-9c144559a51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:28.186386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111a29d-c999-4c43-82d0-9c144559a51a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:28.196549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 366/729] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:28.205051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111a29d-c999-4c43-82d0-9c144559a51a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:28.229261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:28.334169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111a29d-c999-4c43-82d0-9c144559a51a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:28.336034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9f79-69f6-47fc-bad3-eb963d946c54 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:28.393458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321eb8eb-d1ab-49b5-beb4-582f00a649ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:28.394130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321eb8eb-d1ab-49b5-beb4-582f00a649ad None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.394413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321eb8eb-d1ab-49b5-beb4-582f00a649ad None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.394686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321eb8eb-d1ab-49b5-beb4-582f00a649ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.406895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321eb8eb-d1ab-49b5-beb4-582f00a649ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:28.408384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321eb8eb-d1ab-49b5-beb4-582f00a649ad None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:28.410935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f9484-c5d8-4558-aa45-0fbee44efc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:28.411603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f9484-c5d8-4558-aa45-0fbee44efc08 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.411937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f9484-c5d8-4558-aa45-0fbee44efc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.412166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f9484-c5d8-4558-aa45-0fbee44efc08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-321eb8eb-d1ab-49b5-beb4-582f00a649ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:28.435652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:28.436709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:28.438292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 367/730] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:05:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:28.438882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f9484-c5d8-4558-aa45-0fbee44efc08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:28.441901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f9484-c5d8-4558-aa45-0fbee44efc08 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:28.447029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321eb8eb-d1ab-49b5-beb4-582f00a649ad None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4a3f9484-c5d8-4558-aa45-0fbee44efc08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:28.462662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:28.464924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:28.467100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:28.467100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:28.467100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:28.467100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 364/731] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:28.473412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f9484-c5d8-4558-aa45-0fbee44efc08 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:28.564875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:28.588655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321eb8eb-d1ab-49b5-beb4-582f00a649ad None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:28.594126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:28.611740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f9484-c5d8-4558-aa45-0fbee44efc08 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:28.626611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af979ba2-735c-4adf-9a17-36adfa1be63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:28.627349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af979ba2-735c-4adf-9a17-36adfa1be63f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.627705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af979ba2-735c-4adf-9a17-36adfa1be63f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.628056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af979ba2-735c-4adf-9a17-36adfa1be63f None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.628880 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af979ba2-735c-4adf-9a17-36adfa1be63f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:28.629262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af979ba2-735c-4adf-9a17-36adfa1be63f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:28.630851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af979ba2-735c-4adf-9a17-36adfa1be63f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:28.651762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 368/732] 10.222.0.83 () {56 vars in 1149 bytes} [Wed Aug 24 11:05:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:28.656246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde00ba7-ec4a-4aa4-8db3-b1b238b97275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:28.656246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde00ba7-ec4a-4aa4-8db3-b1b238b97275 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.656246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde00ba7-ec4a-4aa4-8db3-b1b238b97275 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.656887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733306f-31a5-45f1-9f86-2c0df9225bc2 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.657039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde00ba7-ec4a-4aa4-8db3-b1b238b97275 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.657195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733306f-31a5-45f1-9f86-2c0df9225bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.657195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733306f-31a5-45f1-9f86-2c0df9225bc2 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.657485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde00ba7-ec4a-4aa4-8db3-b1b238b97275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:28.657485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde00ba7-ec4a-4aa4-8db3-b1b238b97275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:28.657485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde00ba7-ec4a-4aa4-8db3-b1b238b97275 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:28.658274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 369/733] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:28.660541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09477859-5f24-4ec1-a360-b68a456a9318 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.660820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09477859-5f24-4ec1-a360-b68a456a9318 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.661095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09477859-5f24-4ec1-a360-b68a456a9318 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.662608 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 370/734] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:28.663262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09477859-5f24-4ec1-a360-b68a456a9318 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:28.672985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 365/735] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:28.678111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48c314-7b11-4850-8c4f-eea090368b2a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.678444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48c314-7b11-4850-8c4f-eea090368b2a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.678654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48c314-7b11-4850-8c4f-eea090368b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.698699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4e48c314-7b11-4850-8c4f-eea090368b2a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:28.699827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4e48c314-7b11-4850-8c4f-eea090368b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:28.775125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 366/736] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:28.776468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1b371-614b-4c99-b6ec-edc83b984c64 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.776760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1b371-614b-4c99-b6ec-edc83b984c64 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.777035 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1b371-614b-4c99-b6ec-edc83b984c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.786478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09477859-5f24-4ec1-a360-b68a456a9318 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:28.800793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6bd1b371-614b-4c99-b6ec-edc83b984c64 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:28.802039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6bd1b371-614b-4c99-b6ec-edc83b984c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:28.836654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b6743-9472-465b-91bc-2f4827ff70fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:28.837925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b6743-9472-465b-91bc-2f4827ff70fc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:28.838431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b6743-9472-465b-91bc-2f4827ff70fc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:28.838912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b6743-9472-465b-91bc-2f4827ff70fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:28.844158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:28.878780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 367/737] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:05:28.880091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1b371-614b-4c99-b6ec-edc83b984c64 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:28.902305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:28.956992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b6743-9472-465b-91bc-2f4827ff70fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:28.958635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b6743-9472-465b-91bc-2f4827ff70fc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:29.080504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 371/738] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:05:29.081388 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b6743-9472-465b-91bc-2f4827ff70fc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:29.104691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1b371-614b-4c99-b6ec-edc83b984c64 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:29.129559 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:29.146657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee2139-c823-4752-9382-90fcf3d365c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:29.147375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee2139-c823-4752-9382-90fcf3d365c9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:29.147910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee2139-c823-4752-9382-90fcf3d365c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:29.148183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee2139-c823-4752-9382-90fcf3d365c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:29.157509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee2139-c823-4752-9382-90fcf3d365c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:29.159695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee2139-c823-4752-9382-90fcf3d365c9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-94ee2139-c823-4752-9382-90fcf3d365c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:29.175942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:29.177163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:29.178171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:29.180555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 368/739] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:29.181746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee2139-c823-4752-9382-90fcf3d365c9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:29.213426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b6743-9472-465b-91bc-2f4827ff70fc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:29.253950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d47e1-ce4d-4a99-84f2-bc48b8126e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:29.254650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d47e1-ce4d-4a99-84f2-bc48b8126e94 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:29.254921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d47e1-ce4d-4a99-84f2-bc48b8126e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:29.255172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d47e1-ce4d-4a99-84f2-bc48b8126e94 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:29.256223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:29.257107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d47e1-ce4d-4a99-84f2-bc48b8126e94 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:29.259187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d47e1-ce4d-4a99-84f2-bc48b8126e94 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:29.292551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee2139-c823-4752-9382-90fcf3d365c9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:29.328435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693ca9d-ea9c-43ec-8841-9f07353df999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:29.329099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693ca9d-ea9c-43ec-8841-9f07353df999 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:29.329383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693ca9d-ea9c-43ec-8841-9f07353df999 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:29.329662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693ca9d-ea9c-43ec-8841-9f07353df999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:29.341076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693ca9d-ea9c-43ec-8841-9f07353df999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:29.342553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693ca9d-ea9c-43ec-8841-9f07353df999 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c693ca9d-ea9c-43ec-8841-9f07353df999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:29.357892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:29.360200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:29.361102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:29.361102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:29.361102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:29.361102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 369/740] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:29.361102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693ca9d-ea9c-43ec-8841-9f07353df999 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:29.433815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 372/741] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:05:29 2022] POST /identity/v3/services => generated 240 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:29.434698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d47e1-ce4d-4a99-84f2-bc48b8126e94 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:29.495142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693ca9d-ea9c-43ec-8841-9f07353df999 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:29.519802 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:29.567731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8709e2-ec9c-47bb-b130-d9e10db315d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:29.568845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8709e2-ec9c-47bb-b130-d9e10db315d6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:29.569361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8709e2-ec9c-47bb-b130-d9e10db315d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:29.569969 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8709e2-ec9c-47bb-b130-d9e10db315d6 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:29.570955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8709e2-ec9c-47bb-b130-d9e10db315d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:29.571472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8709e2-ec9c-47bb-b130-d9e10db315d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:29.573350 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8709e2-ec9c-47bb-b130-d9e10db315d6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:29.579236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:29.588396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d47e1-ce4d-4a99-84f2-bc48b8126e94 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:29.597961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 370/742] 10.222.0.83 () {56 vars in 1135 bytes} [Wed Aug 24 11:05:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:29.599249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8709e2-ec9c-47bb-b130-d9e10db315d6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:29.636071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173b658-d86d-45eb-af0d-27551fa08d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:29.636702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173b658-d86d-45eb-af0d-27551fa08d62 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:29.636981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173b658-d86d-45eb-af0d-27551fa08d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:29.637261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173b658-d86d-45eb-af0d-27551fa08d62 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:29.637988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173b658-d86d-45eb-af0d-27551fa08d62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:29.639463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173b658-d86d-45eb-af0d-27551fa08d62 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:29.727868 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 373/743] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:05:29 2022] POST /identity/v3/services => generated 251 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:29.729263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173b658-d86d-45eb-af0d-27551fa08d62 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:29.743714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8709e2-ec9c-47bb-b130-d9e10db315d6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:29.809957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd23a6-26d6-457e-beb0-1067d2acb309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:29.810586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd23a6-26d6-457e-beb0-1067d2acb309 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:29.810864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd23a6-26d6-457e-beb0-1067d2acb309 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:29.811137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd23a6-26d6-457e-beb0-1067d2acb309 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:29.811811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd23a6-26d6-457e-beb0-1067d2acb309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:29.812144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd23a6-26d6-457e-beb0-1067d2acb309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:29.813677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd23a6-26d6-457e-beb0-1067d2acb309 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:29.832496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 371/744] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:29.834067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beaac4c-9c89-40a6-abff-d0cc94502fbd None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:29.834344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beaac4c-9c89-40a6-abff-d0cc94502fbd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:29.834642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beaac4c-9c89-40a6-abff-d0cc94502fbd None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:29.838033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 372/745] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:29.839242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beaac4c-9c89-40a6-abff-d0cc94502fbd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:29.875511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:29.903923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173b658-d86d-45eb-af0d-27551fa08d62 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:29.954493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817d4f0-3651-46d1-a175-1846cf6023ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:29.955168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817d4f0-3651-46d1-a175-1846cf6023ba None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:29.955748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817d4f0-3651-46d1-a175-1846cf6023ba None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:29.955748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817d4f0-3651-46d1-a175-1846cf6023ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:29.989011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:29.991735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beaac4c-9c89-40a6-abff-d0cc94502fbd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:30.040336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec8152-9f6d-4588-b352-5aa8ee869ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:30.041226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec8152-9f6d-4588-b352-5aa8ee869ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:30.041630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec8152-9f6d-4588-b352-5aa8ee869ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:30.042076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec8152-9f6d-4588-b352-5aa8ee869ed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:30.050161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817d4f0-3651-46d1-a175-1846cf6023ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:30.050593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec8152-9f6d-4588-b352-5aa8ee869ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:30.051748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817d4f0-3651-46d1-a175-1846cf6023ba None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:30.052307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec8152-9f6d-4588-b352-5aa8ee869ed9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b4ec8152-9f6d-4588-b352-5aa8ee869ed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:30.068587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:30.069584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:30.070509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:30.070509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:30.070509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:30.070509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:30.070509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:30.070509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:30.070509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:30.070509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:30.074059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 373/746] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:30.075746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7284-742e-4f88-9f14-b311723c977b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:30.076036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7284-742e-4f88-9f14-b311723c977b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:30.076312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7284-742e-4f88-9f14-b311723c977b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:30.100170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1e0e7284-742e-4f88-9f14-b311723c977b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:30.103130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1e0e7284-742e-4f88-9f14-b311723c977b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:30.189222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 374/747] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:05:30.190253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817d4f0-3651-46d1-a175-1846cf6023ba None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:30.201578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 374/748] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:30.203447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e7284-742e-4f88-9f14-b311723c977b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:30.235579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:30.308823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:30.357129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817d4f0-3651-46d1-a175-1846cf6023ba None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:30.384152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e7284-742e-4f88-9f14-b311723c977b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:30.397687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d8c6a-bf10-4625-bbd7-0856318a43b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:30.400669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d8c6a-bf10-4625-bbd7-0856318a43b1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:30.400951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d8c6a-bf10-4625-bbd7-0856318a43b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:30.401225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d8c6a-bf10-4625-bbd7-0856318a43b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:30.401982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d8c6a-bf10-4625-bbd7-0856318a43b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:30.402287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d8c6a-bf10-4625-bbd7-0856318a43b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:30.403755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d8c6a-bf10-4625-bbd7-0856318a43b1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:30.429852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 375/749] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:30.436640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d8c6a-bf10-4625-bbd7-0856318a43b1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:30.460393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0d8a-f3e0-4e74-b09c-f2a09e41f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:30.461423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0d8a-f3e0-4e74-b09c-f2a09e41f250 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:30.461970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0d8a-f3e0-4e74-b09c-f2a09e41f250 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:30.462441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0d8a-f3e0-4e74-b09c-f2a09e41f250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:30.479121 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd0d8a-f3e0-4e74-b09c-f2a09e41f250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:30.489919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd0d8a-f3e0-4e74-b09c-f2a09e41f250 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b6fd0d8a-f3e0-4e74-b09c-f2a09e41f250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:30.516302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:30.517297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:30.518370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:30.518370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:30.518370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:30.520576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 375/750] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:30.526018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0d8a-f3e0-4e74-b09c-f2a09e41f250 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:30.586459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d8c6a-bf10-4625-bbd7-0856318a43b1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:30.610728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:30.636131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739400c-b018-4dd9-a529-7ad56d46b34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:30.637157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739400c-b018-4dd9-a529-7ad56d46b34a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:30.637634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739400c-b018-4dd9-a529-7ad56d46b34a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:30.638342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739400c-b018-4dd9-a529-7ad56d46b34a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:30.639730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739400c-b018-4dd9-a529-7ad56d46b34a None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=f9d0b6662bf947cba853388819a933b8) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:30.640814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739400c-b018-4dd9-a529-7ad56d46b34a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:30.643044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739400c-b018-4dd9-a529-7ad56d46b34a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:30.651386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 376/751] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:30 2022] GET /identity/v3/role_assignments?user.id=f9d0b6662bf947cba853388819a933b8&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:30.652256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:30.654672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739400c-b018-4dd9-a529-7ad56d46b34a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:30.695786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0d8a-f3e0-4e74-b09c-f2a09e41f250 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:30.772775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fe2c8-b465-475d-b9a7-e89ba5f7281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:30.776176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fe2c8-b465-475d-b9a7-e89ba5f7281a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:30.776454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fe2c8-b465-475d-b9a7-e89ba5f7281a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:30.776730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fe2c8-b465-475d-b9a7-e89ba5f7281a None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:30.777418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fe2c8-b465-475d-b9a7-e89ba5f7281a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:30.777725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fe2c8-b465-475d-b9a7-e89ba5f7281a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:30.779186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fe2c8-b465-475d-b9a7-e89ba5f7281a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:30.812921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 376/752] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:30.815775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fe2c8-b465-475d-b9a7-e89ba5f7281a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:30.841114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739400c-b018-4dd9-a529-7ad56d46b34a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:30.894296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fe2e3-1df6-4a03-a1a7-e004564dab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:30.894953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fe2e3-1df6-4a03-a1a7-e004564dab0e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:30.895236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fe2e3-1df6-4a03-a1a7-e004564dab0e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:30.895511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fe2e3-1df6-4a03-a1a7-e004564dab0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:30.921775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fe2e3-1df6-4a03-a1a7-e004564dab0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-fa6fe2e3-1df6-4a03-a1a7-e004564dab0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:30.930782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:30.931806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:30.932950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:30.941478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 377/753] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:30.947510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fe2e3-1df6-4a03-a1a7-e004564dab0e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:31.007681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fe2c8-b465-475d-b9a7-e89ba5f7281a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:31.034175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:31.077143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:31.106716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cff6ab-f2d2-4b23-ab3c-d82e8827f7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:31.107293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cff6ab-f2d2-4b23-ab3c-d82e8827f7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:31.107575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cff6ab-f2d2-4b23-ab3c-d82e8827f7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:31.107846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cff6ab-f2d2-4b23-ab3c-d82e8827f7d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:31.108486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fe2e3-1df6-4a03-a1a7-e004564dab0e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:31.121393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cff6ab-f2d2-4b23-ab3c-d82e8827f7d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:31.121393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cff6ab-f2d2-4b23-ab3c-d82e8827f7d5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-18cff6ab-f2d2-4b23-ab3c-d82e8827f7d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:31.134459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:31.141749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:31.142899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:31.142899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:31.142899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:31.142899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:31.142899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:31.142899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:31.142899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:31.142899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:31.142899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 377/754] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:31.145988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cff6ab-f2d2-4b23-ab3c-d82e8827f7d5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:31.222245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefc1d2-9d8c-4ba7-9926-472ce148dd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:31.224422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefc1d2-9d8c-4ba7-9926-472ce148dd89 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:31.224732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefc1d2-9d8c-4ba7-9926-472ce148dd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:31.225017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefc1d2-9d8c-4ba7-9926-472ce148dd89 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:31.227423 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefc1d2-9d8c-4ba7-9926-472ce148dd89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:31.227828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefc1d2-9d8c-4ba7-9926-472ce148dd89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:31.232210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefc1d2-9d8c-4ba7-9926-472ce148dd89 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:31.254010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 378/755] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:31.261204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefc1d2-9d8c-4ba7-9926-472ce148dd89 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:31.337544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cff6ab-f2d2-4b23-ab3c-d82e8827f7d5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:31.379745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540184b-b294-4fb3-901d-1a9d13ac7f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:31.380470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540184b-b294-4fb3-901d-1a9d13ac7f36 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:31.380754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540184b-b294-4fb3-901d-1a9d13ac7f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:31.381030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540184b-b294-4fb3-901d-1a9d13ac7f36 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:31.381740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540184b-b294-4fb3-901d-1a9d13ac7f36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:31.382077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540184b-b294-4fb3-901d-1a9d13ac7f36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:31.383632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540184b-b294-4fb3-901d-1a9d13ac7f36 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:31.397958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 378/756] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:31.403936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540184b-b294-4fb3-901d-1a9d13ac7f36 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:31.471987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefc1d2-9d8c-4ba7-9926-472ce148dd89 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:31.473590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:31.499292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:31.514049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff2890-5b06-44b7-8053-d740c28d3884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:31.514705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff2890-5b06-44b7-8053-d740c28d3884 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:31.514988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff2890-5b06-44b7-8053-d740c28d3884 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:31.515290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff2890-5b06-44b7-8053-d740c28d3884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:31.529236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff2890-5b06-44b7-8053-d740c28d3884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:31.531183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff2890-5b06-44b7-8053-d740c28d3884 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-31ff2890-5b06-44b7-8053-d740c28d3884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:31.549521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:31.551351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:31.553764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:31.553764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:31.553764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:31.553764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 379/757] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:31.557422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff2890-5b06-44b7-8053-d740c28d3884 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:31.607934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540184b-b294-4fb3-901d-1a9d13ac7f36 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:31.649012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398a5db-44fd-4469-ae3c-1b9c9ec16dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:31.649695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398a5db-44fd-4469-ae3c-1b9c9ec16dca None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:31.650015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398a5db-44fd-4469-ae3c-1b9c9ec16dca None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:31.650291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398a5db-44fd-4469-ae3c-1b9c9ec16dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:31.651060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398a5db-44fd-4469-ae3c-1b9c9ec16dca None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=12eafcbf5c98425782138d5363db9a25) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:31.651375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398a5db-44fd-4469-ae3c-1b9c9ec16dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:31.652967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398a5db-44fd-4469-ae3c-1b9c9ec16dca None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:31.659747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 379/758] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:31 2022] GET /identity/v3/role_assignments?user.id=12eafcbf5c98425782138d5363db9a25&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:31.667165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e1adc-6f3d-4f1a-96ed-a7b8ed25fb3c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:31.667501 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e1adc-6f3d-4f1a-96ed-a7b8ed25fb3c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:31.667716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e1adc-6f3d-4f1a-96ed-a7b8ed25fb3c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:31.669681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 380/759] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:31.675587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff746db5-a352-4758-a2cb-82c905665182 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:31.676248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff746db5-a352-4758-a2cb-82c905665182 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:31.676748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff746db5-a352-4758-a2cb-82c905665182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:31.679675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff2890-5b06-44b7-8053-d740c28d3884 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:31.697077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ff746db5-a352-4758-a2cb-82c905665182 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:31.698791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ff746db5-a352-4758-a2cb-82c905665182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:31.722021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d064e-10a6-4f49-93ee-c60c5e02787e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:31.722661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d064e-10a6-4f49-93ee-c60c5e02787e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:31.722945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d064e-10a6-4f49-93ee-c60c5e02787e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:31.723224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d064e-10a6-4f49-93ee-c60c5e02787e None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:31.723904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d064e-10a6-4f49-93ee-c60c5e02787e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:31.724213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d064e-10a6-4f49-93ee-c60c5e02787e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:31.725731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d064e-10a6-4f49-93ee-c60c5e02787e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:31.739998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 380/760] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:31.745795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d064e-10a6-4f49-93ee-c60c5e02787e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:31.764355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:31.859823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 381/761] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:05:31.863736 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d064e-10a6-4f49-93ee-c60c5e02787e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:31.904206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2786da5-820a-4fe4-9bab-881edf1e54ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:31.904831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2786da5-820a-4fe4-9bab-881edf1e54ae None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:31.905117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2786da5-820a-4fe4-9bab-881edf1e54ae None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:31.905388 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2786da5-820a-4fe4-9bab-881edf1e54ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:31.913912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2786da5-820a-4fe4-9bab-881edf1e54ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:31.915576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2786da5-820a-4fe4-9bab-881edf1e54ae None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-c2786da5-820a-4fe4-9bab-881edf1e54ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:31.929686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:31.930732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:31.932674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 381/762] 10.222.0.83 () {56 vars in 1093 bytes} [Wed Aug 24 11:05:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:31.937894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff746db5-a352-4758-a2cb-82c905665182 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:31.988498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:32.046350 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff746db5-a352-4758-a2cb-82c905665182 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:32.074622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb784b8-2b3c-42eb-9815-3dc50578bce1 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:32.074942 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb784b8-2b3c-42eb-9815-3dc50578bce1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:32.075167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb784b8-2b3c-42eb-9815-3dc50578bce1 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:32.076967 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 382/763] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:32.084065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb55acc-f025-40f2-965c-812cd628a981 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:32.084441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb55acc-f025-40f2-965c-812cd628a981 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:32.084725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb55acc-f025-40f2-965c-812cd628a981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:32.095283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce329811-9392-4d38-b424-1d19a79e8a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:32.108201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce329811-9392-4d38-b424-1d19a79e8a99 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:32.108201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce329811-9392-4d38-b424-1d19a79e8a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:32.108201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce329811-9392-4d38-b424-1d19a79e8a99 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:32.108201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce329811-9392-4d38-b424-1d19a79e8a99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:32.108201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce329811-9392-4d38-b424-1d19a79e8a99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:32.108201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce329811-9392-4d38-b424-1d19a79e8a99 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:32.108644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ecb55acc-f025-40f2-965c-812cd628a981 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:32.108644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ecb55acc-f025-40f2-965c-812cd628a981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:32.134966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 382/764] 10.222.0.83 () {56 vars in 1133 bytes} [Wed Aug 24 11:05:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:32.142201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce329811-9392-4d38-b424-1d19a79e8a99 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:32.189668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 383/765] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:32.196093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb55acc-f025-40f2-965c-812cd628a981 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:32.200785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:32.314436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce329811-9392-4d38-b424-1d19a79e8a99 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:32.316930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:32.387960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5014-2091-4e62-b835-1c2fa48046a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:32.388604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5014-2091-4e62-b835-1c2fa48046a0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:32.388936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5014-2091-4e62-b835-1c2fa48046a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:32.389215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5014-2091-4e62-b835-1c2fa48046a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:32.408773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af5014-2091-4e62-b835-1c2fa48046a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:32.411050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af5014-2091-4e62-b835-1c2fa48046a0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:32.413935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb55acc-f025-40f2-965c-812cd628a981 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-06af5014-2091-4e62-b835-1c2fa48046a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:32.433468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:32.434504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:32.435386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:32.435386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:32.435386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:32.436803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 383/766] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:32.444250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5014-2091-4e62-b835-1c2fa48046a0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:32.494654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a405fb-1062-457b-ba6d-deba1fad8699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:32.495310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a405fb-1062-457b-ba6d-deba1fad8699 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:32.495587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a405fb-1062-457b-ba6d-deba1fad8699 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:32.495861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a405fb-1062-457b-ba6d-deba1fad8699 None admin] PATH_INFO: `/v3/services/network` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:32.496470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a405fb-1062-457b-ba6d-deba1fad8699 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:32.497886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a405fb-1062-457b-ba6d-deba1fad8699 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-30a405fb-1062-457b-ba6d-deba1fad8699 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:32.514482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:32.516097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:05:32.516916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:32.520187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 384/767] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:32 2022] GET /identity/v3/services/network => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:32.528379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a405fb-1062-457b-ba6d-deba1fad8699 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:32.596472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5014-2091-4e62-b835-1c2fa48046a0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:32.622289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:32.646540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b81591-f2dd-4f22-bbee-6d77a04b7efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:32.647613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b81591-f2dd-4f22-bbee-6d77a04b7efe None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:32.648092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b81591-f2dd-4f22-bbee-6d77a04b7efe None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:32.648560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b81591-f2dd-4f22-bbee-6d77a04b7efe None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:32.649461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b81591-f2dd-4f22-bbee-6d77a04b7efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:32.649992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b81591-f2dd-4f22-bbee-6d77a04b7efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:32.651583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b81591-f2dd-4f22-bbee-6d77a04b7efe None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:32.663542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a405fb-1062-457b-ba6d-deba1fad8699 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:32.665118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 384/768] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:32.667341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b81591-f2dd-4f22-bbee-6d77a04b7efe None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:32.684902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:32.708529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd3f96-1ce1-46ee-adbb-aa87d3fd2cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:32.709065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd3f96-1ce1-46ee-adbb-aa87d3fd2cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:32.709343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd3f96-1ce1-46ee-adbb-aa87d3fd2cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:32.709623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd3f96-1ce1-46ee-adbb-aa87d3fd2cf0 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:32.710393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd3f96-1ce1-46ee-adbb-aa87d3fd2cf0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:32.710703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd3f96-1ce1-46ee-adbb-aa87d3fd2cf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:32.712251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd3f96-1ce1-46ee-adbb-aa87d3fd2cf0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:32.717705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 385/769] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:05:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:32.718877 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd3f96-1ce1-46ee-adbb-aa87d3fd2cf0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:32.783012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b81591-f2dd-4f22-bbee-6d77a04b7efe None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:32.821534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1145d82-a572-4b85-856a-fdb88f359458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:32.822640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1145d82-a572-4b85-856a-fdb88f359458 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:32.823119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1145d82-a572-4b85-856a-fdb88f359458 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:32.823587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1145d82-a572-4b85-856a-fdb88f359458 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:32.824431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1145d82-a572-4b85-856a-fdb88f359458 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:32.826278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1145d82-a572-4b85-856a-fdb88f359458 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:32.831722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd3f96-1ce1-46ee-adbb-aa87d3fd2cf0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b1145d82-a572-4b85-856a-fdb88f359458 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:32.836470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:32.837465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:05:32.838657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:32.841272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 385/770] 10.222.0.83 () {56 vars in 1129 bytes} [Wed Aug 24 11:05:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:32.842452 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1145d82-a572-4b85-856a-fdb88f359458 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:32.865587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:32.872181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab93b28-fa1b-4a29-acf9-e80c04ed4c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:32.872832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab93b28-fa1b-4a29-acf9-e80c04ed4c6d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:32.873113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab93b28-fa1b-4a29-acf9-e80c04ed4c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:32.873393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab93b28-fa1b-4a29-acf9-e80c04ed4c6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:32.882233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab93b28-fa1b-4a29-acf9-e80c04ed4c6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:32.883873 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab93b28-fa1b-4a29-acf9-e80c04ed4c6d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3ab93b28-fa1b-4a29-acf9-e80c04ed4c6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:32.901196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:32.902185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:32.903064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:32.903064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:32.903064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:32.903064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:32.903064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:32.903064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:32.903064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:32.903064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:32.910452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 386/771] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:32.911668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab93b28-fa1b-4a29-acf9-e80c04ed4c6d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:32.940882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:32.964700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1145d82-a572-4b85-856a-fdb88f359458 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:33.005280 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac68ae-8136-40c2-b296-975eb8016a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:33.006292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac68ae-8136-40c2-b296-975eb8016a59 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.006797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac68ae-8136-40c2-b296-975eb8016a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.007267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac68ae-8136-40c2-b296-975eb8016a59 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.008124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac68ae-8136-40c2-b296-975eb8016a59 None admin] RBAC: Adding query filter params (type=network) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:33.008696 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac68ae-8136-40c2-b296-975eb8016a59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:33.010581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac68ae-8136-40c2-b296-975eb8016a59 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:33.018832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 386/772] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:05:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:33.020706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac68ae-8136-40c2-b296-975eb8016a59 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:33.043745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab93b28-fa1b-4a29-acf9-e80c04ed4c6d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:33.085072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa19d38-375d-4673-8627-dc7e0a50dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:33.085741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa19d38-375d-4673-8627-dc7e0a50dd51 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.086052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa19d38-375d-4673-8627-dc7e0a50dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.086331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa19d38-375d-4673-8627-dc7e0a50dd51 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.086975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa19d38-375d-4673-8627-dc7e0a50dd51 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:33.087278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa19d38-375d-4673-8627-dc7e0a50dd51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:33.088688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa19d38-375d-4673-8627-dc7e0a50dd51 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:33.094767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 387/773] 10.222.0.83 () {56 vars in 1124 bytes} [Wed Aug 24 11:05:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:33.095659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa19d38-375d-4673-8627-dc7e0a50dd51 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:33.142225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac68ae-8136-40c2-b296-975eb8016a59 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:33.159703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:33.181107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bcd68-2e85-47ec-99ea-dc4de5e8ee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:33.181759 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bcd68-2e85-47ec-99ea-dc4de5e8ee96 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.182294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bcd68-2e85-47ec-99ea-dc4de5e8ee96 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.182568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bcd68-2e85-47ec-99ea-dc4de5e8ee96 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.183271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bcd68-2e85-47ec-99ea-dc4de5e8ee96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:33.183716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bcd68-2e85-47ec-99ea-dc4de5e8ee96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:33.185159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bcd68-2e85-47ec-99ea-dc4de5e8ee96 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:33.197965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 387/774] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:33.204325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8496a3d-2959-4f40-8ce1-2509766ef260 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.204617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8496a3d-2959-4f40-8ce1-2509766ef260 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.204994 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8496a3d-2959-4f40-8ce1-2509766ef260 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.210793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 388/775] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:33.210793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8496a3d-2959-4f40-8ce1-2509766ef260 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:33.210793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:33.216171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa19d38-375d-4673-8627-dc7e0a50dd51 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:33.259446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26618a-c459-4274-895c-28cfc772a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:33.260646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26618a-c459-4274-895c-28cfc772a16d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.261058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26618a-c459-4274-895c-28cfc772a16d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.261467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26618a-c459-4274-895c-28cfc772a16d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.262596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26618a-c459-4274-895c-28cfc772a16d None admin] RBAC: Adding query filter params (interface=public, service_id=f225a6f0c318491fa2c314596a1295dc, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:33.263072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26618a-c459-4274-895c-28cfc772a16d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:33.265521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26618a-c459-4274-895c-28cfc772a16d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:33.280158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 388/776] 10.222.0.83 () {56 vars in 1251 bytes} [Wed Aug 24 11:05:33 2022] GET /identity/v3/endpoints?service_id=f225a6f0c318491fa2c314596a1295dc&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:33.280158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26618a-c459-4274-895c-28cfc772a16d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:33.334347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8496a3d-2959-4f40-8ce1-2509766ef260 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:33.377849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d6446-3629-4aa2-9e6b-d54ff32e879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:33.378478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d6446-3629-4aa2-9e6b-d54ff32e879e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.378757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d6446-3629-4aa2-9e6b-d54ff32e879e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.379030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d6446-3629-4aa2-9e6b-d54ff32e879e None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.379665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d6446-3629-4aa2-9e6b-d54ff32e879e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:33.379967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d6446-3629-4aa2-9e6b-d54ff32e879e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:33.381343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d6446-3629-4aa2-9e6b-d54ff32e879e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:33.386890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 389/777] 10.222.0.83 () {56 vars in 1124 bytes} [Wed Aug 24 11:05:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:33.388402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ff8b8-28a8-4970-83d4-603df68a69ff None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.388752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ff8b8-28a8-4970-83d4-603df68a69ff None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.389081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ff8b8-28a8-4970-83d4-603df68a69ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.408735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d09ff8b8-28a8-4970-83d4-603df68a69ff None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:33.409967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d09ff8b8-28a8-4970-83d4-603df68a69ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:33.412649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26618a-c459-4274-895c-28cfc772a16d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:33.436864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:33.453484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee57737-a315-49a1-b1a7-c94085ea2dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:33.454104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee57737-a315-49a1-b1a7-c94085ea2dda None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.478367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee57737-a315-49a1-b1a7-c94085ea2dda None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.478367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee57737-a315-49a1-b1a7-c94085ea2dda None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/32a28f9fb3134f33819e53e0c72c63b4/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.478367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee57737-a315-49a1-b1a7-c94085ea2dda None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=32a28f9fb3134f33819e53e0c72c63b4, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:33.478367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee57737-a315-49a1-b1a7-c94085ea2dda None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:33.478749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:33.486022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 390/778] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:33.487062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ff8b8-28a8-4970-83d4-603df68a69ff None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:33.588182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 389/779] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:05:33 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/32a28f9fb3134f33819e53e0c72c63b4/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:05:33.590086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee57737-a315-49a1-b1a7-c94085ea2dda None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:33.641060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ff8b8-28a8-4970-83d4-603df68a69ff None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:33.741946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee57737-a315-49a1-b1a7-c94085ea2dda None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:33.747739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c25d05-f7af-4eeb-bb09-a0b7db495e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:33.748376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c25d05-f7af-4eeb-bb09-a0b7db495e05 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.748655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c25d05-f7af-4eeb-bb09-a0b7db495e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.748925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c25d05-f7af-4eeb-bb09-a0b7db495e05 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.749606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c25d05-f7af-4eeb-bb09-a0b7db495e05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:33.751051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c25d05-f7af-4eeb-bb09-a0b7db495e05 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:33.772955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 391/780] 10.222.0.83 () {56 vars in 1087 bytes} [Wed Aug 24 11:05:33 2022] GET /identity/v3/services => generated 772 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:33.789904 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c25d05-f7af-4eeb-bb09-a0b7db495e05 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:33.802805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d7454-b1eb-4d97-9a01-a3fa200da65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:33.803458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d7454-b1eb-4d97-9a01-a3fa200da65e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:33.803743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d7454-b1eb-4d97-9a01-a3fa200da65e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:33.804020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d7454-b1eb-4d97-9a01-a3fa200da65e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:33.804779 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d7454-b1eb-4d97-9a01-a3fa200da65e None admin] RBAC: Adding query filter params (interface=public, service_id=2a8db5cf682c421ebcfb73003c34ae60, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:33.805095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d7454-b1eb-4d97-9a01-a3fa200da65e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:33.806591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d7454-b1eb-4d97-9a01-a3fa200da65e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:33.817000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 390/781] 10.222.0.83 () {56 vars in 1251 bytes} [Wed Aug 24 11:05:33 2022] GET /identity/v3/endpoints?service_id=2a8db5cf682c421ebcfb73003c34ae60&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:33.823413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d7454-b1eb-4d97-9a01-a3fa200da65e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:33.906857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:33.947411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:33.958201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d7454-b1eb-4d97-9a01-a3fa200da65e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:34.004285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47fab1-0687-4d95-8b84-ee9a7f24f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:34.004839 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47fab1-0687-4d95-8b84-ee9a7f24f86b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:34.005459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47fab1-0687-4d95-8b84-ee9a7f24f86b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:34.005744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47fab1-0687-4d95-8b84-ee9a7f24f86b None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:34.006406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47fab1-0687-4d95-8b84-ee9a7f24f86b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:34.008365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47fab1-0687-4d95-8b84-ee9a7f24f86b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:34.014612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 391/782] 10.222.0.83 () {56 vars in 1087 bytes} [Wed Aug 24 11:05:33 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:34.016677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ea07b-2651-4b99-bd6a-2f6907fa7987 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:34.016677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ea07b-2651-4b99-bd6a-2f6907fa7987 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:34.016677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ea07b-2651-4b99-bd6a-2f6907fa7987 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:34.018019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 392/783] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:34.024440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500466c8-6aa6-41a1-bb84-786a057993fc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:34.024766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500466c8-6aa6-41a1-bb84-786a057993fc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:34.024994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500466c8-6aa6-41a1-bb84-786a057993fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:34.034495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c25d05-f7af-4eeb-bb09-a0b7db495e05 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:34.049100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-500466c8-6aa6-41a1-bb84-786a057993fc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:34.051199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-500466c8-6aa6-41a1-bb84-786a057993fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:34.086978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519cfa8-e91e-4d49-8134-62edf0f7e367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:34.088792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519cfa8-e91e-4d49-8134-62edf0f7e367 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:34.089597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519cfa8-e91e-4d49-8134-62edf0f7e367 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:34.090155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519cfa8-e91e-4d49-8134-62edf0f7e367 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:34.106944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519cfa8-e91e-4d49-8134-62edf0f7e367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4519cfa8-e91e-4d49-8134-62edf0f7e367 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:34.113704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:34.121460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:34.122384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 392/784] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:34.124344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519cfa8-e91e-4d49-8134-62edf0f7e367 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:34.150567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 393/785] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:34.272241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519cfa8-e91e-4d49-8134-62edf0f7e367 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:34.298019 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:34.318133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f2de3-487c-40d8-8fc6-47636e980701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:34.318762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f2de3-487c-40d8-8fc6-47636e980701 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:34.319043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f2de3-487c-40d8-8fc6-47636e980701 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:34.319577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f2de3-487c-40d8-8fc6-47636e980701 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:34.320016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f2de3-487c-40d8-8fc6-47636e980701 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:34.320324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f2de3-487c-40d8-8fc6-47636e980701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:34.321746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f2de3-487c-40d8-8fc6-47636e980701 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:34.340146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 393/786] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:34.346502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500466c8-6aa6-41a1-bb84-786a057993fc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:34.408205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:34.478012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500466c8-6aa6-41a1-bb84-786a057993fc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:34.511091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f2de3-487c-40d8-8fc6-47636e980701 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:34.516063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f8e77-cf63-4316-bb63-a8e9a4124772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:34.516989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f8e77-cf63-4316-bb63-a8e9a4124772 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:34.517476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f8e77-cf63-4316-bb63-a8e9a4124772 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:34.517986 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f8e77-cf63-4316-bb63-a8e9a4124772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:34.529495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f8e77-cf63-4316-bb63-a8e9a4124772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:34.531405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f8e77-cf63-4316-bb63-a8e9a4124772 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e71f8e77-cf63-4316-bb63-a8e9a4124772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:34.549380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:34.550547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:34.551644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:34.551644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:34.551644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:34.553376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 394/787] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:34.559409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f8e77-cf63-4316-bb63-a8e9a4124772 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:34.680938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f8e77-cf63-4316-bb63-a8e9a4124772 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:34.710307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:34.723303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad39a2-262a-4cc6-83fb-c26809e7b887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:34.724349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad39a2-262a-4cc6-83fb-c26809e7b887 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:34.724781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad39a2-262a-4cc6-83fb-c26809e7b887 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:34.725285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad39a2-262a-4cc6-83fb-c26809e7b887 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:34.726151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad39a2-262a-4cc6-83fb-c26809e7b887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:34.726818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad39a2-262a-4cc6-83fb-c26809e7b887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:34.729447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad39a2-262a-4cc6-83fb-c26809e7b887 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:34.744067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 395/788] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:34.752978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad39a2-262a-4cc6-83fb-c26809e7b887 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:34.805244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f2de3-487c-40d8-8fc6-47636e980701 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:34.873550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0921e-d67a-452b-8482-ec36025e69a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:34.874215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0921e-d67a-452b-8482-ec36025e69a2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:34.875030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0921e-d67a-452b-8482-ec36025e69a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:34.875305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0921e-d67a-452b-8482-ec36025e69a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:34.886821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0921e-d67a-452b-8482-ec36025e69a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-12d0921e-d67a-452b-8482-ec36025e69a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:34.893414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:34.894440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:34.895263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 394/789] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:34.902441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0921e-d67a-452b-8482-ec36025e69a2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:34.950855 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad39a2-262a-4cc6-83fb-c26809e7b887 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:35.000296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:35.021772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a2e82-6bc0-457b-b6f0-0e9f118525f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:35.022511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a2e82-6bc0-457b-b6f0-0e9f118525f9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:35.022786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a2e82-6bc0-457b-b6f0-0e9f118525f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:35.023093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a2e82-6bc0-457b-b6f0-0e9f118525f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:35.035975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a2e82-6bc0-457b-b6f0-0e9f118525f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:35.040279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a2e82-6bc0-457b-b6f0-0e9f118525f9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-532a2e82-6bc0-457b-b6f0-0e9f118525f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:35.067752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:35.068804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:35.069756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:35.070762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 396/790] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:05:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:35.076482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a2e82-6bc0-457b-b6f0-0e9f118525f9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:35.098271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:35.107160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0921e-d67a-452b-8482-ec36025e69a2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:35.175798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9d3ce-bbbd-4e14-81ef-a2b2671f6821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:35.176531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9d3ce-bbbd-4e14-81ef-a2b2671f6821 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:35.176840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9d3ce-bbbd-4e14-81ef-a2b2671f6821 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:35.177106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9d3ce-bbbd-4e14-81ef-a2b2671f6821 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:35.177818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9d3ce-bbbd-4e14-81ef-a2b2671f6821 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:35.178142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9d3ce-bbbd-4e14-81ef-a2b2671f6821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:35.179537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9d3ce-bbbd-4e14-81ef-a2b2671f6821 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:35.193411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 395/791] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:35.199813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9d3ce-bbbd-4e14-81ef-a2b2671f6821 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:35.242694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a2e82-6bc0-457b-b6f0-0e9f118525f9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:35.314121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f26f5f-9e16-4e12-b2d5-271ab758f44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:35.314960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f26f5f-9e16-4e12-b2d5-271ab758f44d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:35.315235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f26f5f-9e16-4e12-b2d5-271ab758f44d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:35.316780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f26f5f-9e16-4e12-b2d5-271ab758f44d None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:35.317625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f26f5f-9e16-4e12-b2d5-271ab758f44d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:35.318018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f26f5f-9e16-4e12-b2d5-271ab758f44d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:35.320396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f26f5f-9e16-4e12-b2d5-271ab758f44d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:35.360947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 397/792] 10.222.0.83 () {56 vars in 1149 bytes} [Wed Aug 24 11:05:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:35.366513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f26f5f-9e16-4e12-b2d5-271ab758f44d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:35.406383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9d3ce-bbbd-4e14-81ef-a2b2671f6821 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:35.410209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:35.484998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999d6d3-56c9-4558-890a-96a87e02fb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:35.485928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999d6d3-56c9-4558-890a-96a87e02fb84 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:35.486231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:35.486531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999d6d3-56c9-4558-890a-96a87e02fb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:35.486947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999d6d3-56c9-4558-890a-96a87e02fb84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:35.509263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999d6d3-56c9-4558-890a-96a87e02fb84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:35.511005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999d6d3-56c9-4558-890a-96a87e02fb84 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-7999d6d3-56c9-4558-890a-96a87e02fb84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:35.538848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:35.540521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:35.541618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:35.541618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:35.541618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:35.543880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 396/793] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:35.550681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999d6d3-56c9-4558-890a-96a87e02fb84 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:35.558568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f26f5f-9e16-4e12-b2d5-271ab758f44d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:35.619494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b3fe8-d732-48ef-a71f-a48a492778d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:35.620564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b3fe8-d732-48ef-a71f-a48a492778d0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:35.622396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b3fe8-d732-48ef-a71f-a48a492778d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:35.623470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b3fe8-d732-48ef-a71f-a48a492778d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:35.642968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b3fe8-d732-48ef-a71f-a48a492778d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:35.645570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b3fe8-d732-48ef-a71f-a48a492778d0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7f7b3fe8-d732-48ef-a71f-a48a492778d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:35.672772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:35.673985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:35.675071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:35.675071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:35.675071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:35.683742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 398/794] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:35.688784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b3fe8-d732-48ef-a71f-a48a492778d0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:35.759877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999d6d3-56c9-4558-890a-96a87e02fb84 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:35.810460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0090e-5f43-44c9-b594-10782de1d0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:35.811077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0090e-5f43-44c9-b594-10782de1d0ea None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:35.811355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0090e-5f43-44c9-b594-10782de1d0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:35.811633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0090e-5f43-44c9-b594-10782de1d0ea None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:35.812318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0090e-5f43-44c9-b594-10782de1d0ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:35.812624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0090e-5f43-44c9-b594-10782de1d0ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:35.814047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0090e-5f43-44c9-b594-10782de1d0ea None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:35.832580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 397/795] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:35.837907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0090e-5f43-44c9-b594-10782de1d0ea None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:35.869241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:35.910762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b3fe8-d732-48ef-a71f-a48a492778d0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:35.947028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:35.957627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d06d9e-d124-4105-95f0-cdf3a5028b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:35.958281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d06d9e-d124-4105-95f0-cdf3a5028b25 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:35.958560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d06d9e-d124-4105-95f0-cdf3a5028b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:35.958920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d06d9e-d124-4105-95f0-cdf3a5028b25 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:35.959492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d06d9e-d124-4105-95f0-cdf3a5028b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:35.959807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d06d9e-d124-4105-95f0-cdf3a5028b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:35.961284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d06d9e-d124-4105-95f0-cdf3a5028b25 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:35.975460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 399/796] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:35.976903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d48b84-2057-437c-9191-a8062fd61392 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:35.977194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d48b84-2057-437c-9191-a8062fd61392 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:35.977460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d48b84-2057-437c-9191-a8062fd61392 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:35.982900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 400/797] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:35.982900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d48b84-2057-437c-9191-a8062fd61392 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:36.003251 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0090e-5f43-44c9-b594-10782de1d0ea None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:36.047268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367e38a-9997-4975-b376-05db3499fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:36.048644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367e38a-9997-4975-b376-05db3499fe52 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.049340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367e38a-9997-4975-b376-05db3499fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.050057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367e38a-9997-4975-b376-05db3499fe52 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.064318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367e38a-9997-4975-b376-05db3499fe52 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:36.074141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367e38a-9997-4975-b376-05db3499fe52 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c367e38a-9997-4975-b376-05db3499fe52 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:36.084326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:36.085366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:36.086386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:36.093567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 398/798] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:36.095390 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3656ac-243f-4c0d-b254-476fd65ce438 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.095895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3656ac-243f-4c0d-b254-476fd65ce438 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.096366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3656ac-243f-4c0d-b254-476fd65ce438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.102660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d48b84-2057-437c-9191-a8062fd61392 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:36.117587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1c3656ac-243f-4c0d-b254-476fd65ce438 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:36.119278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1c3656ac-243f-4c0d-b254-476fd65ce438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:36.141050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34edd3-9d6c-406c-a07d-7f81f47dab34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:36.141723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34edd3-9d6c-406c-a07d-7f81f47dab34 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.142037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34edd3-9d6c-406c-a07d-7f81f47dab34 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.142309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34edd3-9d6c-406c-a07d-7f81f47dab34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.151128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34edd3-9d6c-406c-a07d-7f81f47dab34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:36.152662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34edd3-9d6c-406c-a07d-7f81f47dab34 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-9d34edd3-9d6c-406c-a07d-7f81f47dab34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:36.166363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:36.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 401/799] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:36.168899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34edd3-9d6c-406c-a07d-7f81f47dab34 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:36.214055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 399/800] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:36.215393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3656ac-243f-4c0d-b254-476fd65ce438 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:36.228361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:36.229463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:36.276980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34edd3-9d6c-406c-a07d-7f81f47dab34 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:36.325254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78057e1f-cb89-4bb3-8305-efd715ec08dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:36.325978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78057e1f-cb89-4bb3-8305-efd715ec08dc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.326268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78057e1f-cb89-4bb3-8305-efd715ec08dc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.326589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78057e1f-cb89-4bb3-8305-efd715ec08dc None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.327550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78057e1f-cb89-4bb3-8305-efd715ec08dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:36.328521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78057e1f-cb89-4bb3-8305-efd715ec08dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:36.330203 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78057e1f-cb89-4bb3-8305-efd715ec08dc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:36.350636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 402/801] 10.222.0.83 () {56 vars in 1135 bytes} [Wed Aug 24 11:05:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:36.351948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38ef13-82c0-4a44-a7b5-aebb3f1dd0db None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.352235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38ef13-82c0-4a44-a7b5-aebb3f1dd0db None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.352513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38ef13-82c0-4a44-a7b5-aebb3f1dd0db None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.358827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 403/802] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:36.359276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38ef13-82c0-4a44-a7b5-aebb3f1dd0db None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:36.383509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3656ac-243f-4c0d-b254-476fd65ce438 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:36.425273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde0aef-1a5a-44c8-874d-02d3e7866a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:36.425714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde0aef-1a5a-44c8-874d-02d3e7866a07 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.426209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde0aef-1a5a-44c8-874d-02d3e7866a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.426589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde0aef-1a5a-44c8-874d-02d3e7866a07 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.427912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde0aef-1a5a-44c8-874d-02d3e7866a07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:36.428391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde0aef-1a5a-44c8-874d-02d3e7866a07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:36.430233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde0aef-1a5a-44c8-874d-02d3e7866a07 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:36.453140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 400/803] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:36.453140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa11bb-4639-4141-b29b-b591b487a3bf None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.453140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa11bb-4639-4141-b29b-b591b487a3bf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.453140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa11bb-4639-4141-b29b-b591b487a3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.472707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:36.474697 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-66fa11bb-4639-4141-b29b-b591b487a3bf None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:36.476024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-66fa11bb-4639-4141-b29b-b591b487a3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:36.477445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38ef13-82c0-4a44-a7b5-aebb3f1dd0db None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:36.515785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88b04a-6a58-4b04-964d-aadefb4252a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:36.516528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88b04a-6a58-4b04-964d-aadefb4252a3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.516687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88b04a-6a58-4b04-964d-aadefb4252a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.516955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88b04a-6a58-4b04-964d-aadefb4252a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.528751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88b04a-6a58-4b04-964d-aadefb4252a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:36.530241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88b04a-6a58-4b04-964d-aadefb4252a3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ce88b04a-6a58-4b04-964d-aadefb4252a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:36.546004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:36.547533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:36.548486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:36.548486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:36.548486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:36.548657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:36.552789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 404/804] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:36.554019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8f7ac-a9f4-4d62-af5e-9ebfd8da9493 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.554391 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8f7ac-a9f4-4d62-af5e-9ebfd8da9493 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.554667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8f7ac-a9f4-4d62-af5e-9ebfd8da9493 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.555991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 405/805] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:36.562621 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 401/806] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:36.562796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8f7ac-a9f4-4d62-af5e-9ebfd8da9493 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:36.568477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa11bb-4639-4141-b29b-b591b487a3bf None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:36.661563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa11bb-4639-4141-b29b-b591b487a3bf None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:36.677485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8f7ac-a9f4-4d62-af5e-9ebfd8da9493 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:36.698531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11547d1-1e68-4aba-9b51-5cc1f28bb909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:36.699134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11547d1-1e68-4aba-9b51-5cc1f28bb909 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.699410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11547d1-1e68-4aba-9b51-5cc1f28bb909 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.699702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11547d1-1e68-4aba-9b51-5cc1f28bb909 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.700377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11547d1-1e68-4aba-9b51-5cc1f28bb909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:36.700775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11547d1-1e68-4aba-9b51-5cc1f28bb909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:36.702065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11547d1-1e68-4aba-9b51-5cc1f28bb909 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:36.714774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 402/807] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:36.715607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11547d1-1e68-4aba-9b51-5cc1f28bb909 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:36.720906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db56c7-0d60-43e1-953b-434484c0684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:36.721726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db56c7-0d60-43e1-953b-434484c0684c None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.722109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db56c7-0d60-43e1-953b-434484c0684c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.722484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db56c7-0d60-43e1-953b-434484c0684c None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/f9d0b6662bf947cba853388819a933b8/roles/7612e1bae5ad4bda9f0675ad732646b4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.739358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db56c7-0d60-43e1-953b-434484c0684c None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=f9d0b6662bf947cba853388819a933b8, role_id=7612e1bae5ad4bda9f0675ad732646b4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:36.741009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db56c7-0d60-43e1-953b-434484c0684c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:36.776725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:36.821230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11547d1-1e68-4aba-9b51-5cc1f28bb909 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:36.830062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:36.834757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 406/808] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:05:36 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/f9d0b6662bf947cba853388819a933b8/roles/7612e1bae5ad4bda9f0675ad732646b4 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:05:36.835990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb5276-9526-4c0e-8556-4be8057145b5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.836276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb5276-9526-4c0e-8556-4be8057145b5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.836545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb5276-9526-4c0e-8556-4be8057145b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.862211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045ca24-d01a-4b87-8dac-8bcf53249b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:36.862853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045ca24-d01a-4b87-8dac-8bcf53249b52 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:36.863132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045ca24-d01a-4b87-8dac-8bcf53249b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:36.863408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045ca24-d01a-4b87-8dac-8bcf53249b52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:36.864644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8dfb5276-9526-4c0e-8556-4be8057145b5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:36.866094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8dfb5276-9526-4c0e-8556-4be8057145b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:36.876197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045ca24-d01a-4b87-8dac-8bcf53249b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c045ca24-d01a-4b87-8dac-8bcf53249b52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:36.881622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:36.882736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:36.883763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 403/809] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:36.884464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045ca24-d01a-4b87-8dac-8bcf53249b52 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:36.962886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 407/810] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:36.963814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb5276-9526-4c0e-8556-4be8057145b5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.021300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045ca24-d01a-4b87-8dac-8bcf53249b52 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.042460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:37.065253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d6f62-dff5-4c02-9218-94768a2cb5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.066006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d6f62-dff5-4c02-9218-94768a2cb5ae None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.066288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d6f62-dff5-4c02-9218-94768a2cb5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.066563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d6f62-dff5-4c02-9218-94768a2cb5ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.074437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d6f62-dff5-4c02-9218-94768a2cb5ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.076190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d6f62-dff5-4c02-9218-94768a2cb5ae None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-dd0d6f62-dff5-4c02-9218-94768a2cb5ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:37.089681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:37.090746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:37.091646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:37.091646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:37.091646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:37.091646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:37.091646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:37.091646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:37.091646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:37.091646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:37.094396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 404/811] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:37.095437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d6f62-dff5-4c02-9218-94768a2cb5ae None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.114053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb5276-9526-4c0e-8556-4be8057145b5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.145727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:37.152887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730adb54-48de-4b45-9ff1-7bcf77726692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.153784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730adb54-48de-4b45-9ff1-7bcf77726692 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.153921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730adb54-48de-4b45-9ff1-7bcf77726692 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.154192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730adb54-48de-4b45-9ff1-7bcf77726692 None admin] PATH_INFO: `/v3/services/network` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.154861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730adb54-48de-4b45-9ff1-7bcf77726692 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.156815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730adb54-48de-4b45-9ff1-7bcf77726692 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-730adb54-48de-4b45-9ff1-7bcf77726692 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:37.166997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:37.168085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:37.168926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 408/812] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:36 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:37.169715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730adb54-48de-4b45-9ff1-7bcf77726692 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.215005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d6f62-dff5-4c02-9218-94768a2cb5ae None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.251933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe740f-dab0-4626-b32b-f66e6b66bf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.252854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe740f-dab0-4626-b32b-f66e6b66bf42 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.253284 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe740f-dab0-4626-b32b-f66e6b66bf42 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.253703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe740f-dab0-4626-b32b-f66e6b66bf42 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.254604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe740f-dab0-4626-b32b-f66e6b66bf42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:37.255059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe740f-dab0-4626-b32b-f66e6b66bf42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.256575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe740f-dab0-4626-b32b-f66e6b66bf42 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:37.267114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 405/813] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:37.268379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe740f-dab0-4626-b32b-f66e6b66bf42 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.292839 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730adb54-48de-4b45-9ff1-7bcf77726692 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.314047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:37.333022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87bdcf-7439-4407-bc33-2c55ecadeb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.333664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87bdcf-7439-4407-bc33-2c55ecadeb15 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.333969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87bdcf-7439-4407-bc33-2c55ecadeb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.334244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87bdcf-7439-4407-bc33-2c55ecadeb15 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.334941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87bdcf-7439-4407-bc33-2c55ecadeb15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:37.335275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87bdcf-7439-4407-bc33-2c55ecadeb15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.336688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87bdcf-7439-4407-bc33-2c55ecadeb15 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:37.350290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 409/814] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:37.351132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87bdcf-7439-4407-bc33-2c55ecadeb15 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.404128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe740f-dab0-4626-b32b-f66e6b66bf42 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.412696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:37.442184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e15ea-f7aa-4f64-a352-3cfd64172ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.442788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e15ea-f7aa-4f64-a352-3cfd64172ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.443070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e15ea-f7aa-4f64-a352-3cfd64172ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.443342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e15ea-f7aa-4f64-a352-3cfd64172ba4 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.443990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e15ea-f7aa-4f64-a352-3cfd64172ba4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:37.444265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e15ea-f7aa-4f64-a352-3cfd64172ba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.445657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e15ea-f7aa-4f64-a352-3cfd64172ba4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:37.455570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 406/815] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:05:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:37.455570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e15ea-f7aa-4f64-a352-3cfd64172ba4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.465665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87bdcf-7439-4407-bc33-2c55ecadeb15 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.510039 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1887d-e62f-474a-b10f-105e7613fc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.510679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1887d-e62f-474a-b10f-105e7613fc1b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.510955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1887d-e62f-474a-b10f-105e7613fc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.511224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1887d-e62f-474a-b10f-105e7613fc1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.523812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1887d-e62f-474a-b10f-105e7613fc1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.525302 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1887d-e62f-474a-b10f-105e7613fc1b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-58b1887d-e62f-474a-b10f-105e7613fc1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:37.542036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:37.543369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:37.544422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:37.544422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:37.544422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:37.544422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 410/816] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:37.544422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1887d-e62f-474a-b10f-105e7613fc1b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.581284 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e15ea-f7aa-4f64-a352-3cfd64172ba4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.583410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:37.619415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373673a-e9e3-43ec-911b-14c213d94427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.620061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373673a-e9e3-43ec-911b-14c213d94427 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.620682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373673a-e9e3-43ec-911b-14c213d94427 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.621177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373673a-e9e3-43ec-911b-14c213d94427 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.622086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373673a-e9e3-43ec-911b-14c213d94427 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.623955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373673a-e9e3-43ec-911b-14c213d94427 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-6373673a-e9e3-43ec-911b-14c213d94427 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:37.637352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:37.638721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:37.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 407/817] 10.222.0.83 () {56 vars in 1129 bytes} [Wed Aug 24 11:05:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:37.641362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373673a-e9e3-43ec-911b-14c213d94427 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.680710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1887d-e62f-474a-b10f-105e7613fc1b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.690743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:37.723443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93378b83-3159-4771-9903-7d1d6ff7db82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.724051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93378b83-3159-4771-9903-7d1d6ff7db82 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.724287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93378b83-3159-4771-9903-7d1d6ff7db82 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.724566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93378b83-3159-4771-9903-7d1d6ff7db82 None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/12eafcbf5c98425782138d5363db9a25/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.726958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93378b83-3159-4771-9903-7d1d6ff7db82 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=12eafcbf5c98425782138d5363db9a25, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.728712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93378b83-3159-4771-9903-7d1d6ff7db82 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:37.752630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373673a-e9e3-43ec-911b-14c213d94427 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.786437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b154f-f491-418e-86cb-6d4007fe707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.787014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b154f-f491-418e-86cb-6d4007fe707f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.787290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b154f-f491-418e-86cb-6d4007fe707f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.787558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b154f-f491-418e-86cb-6d4007fe707f None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.788181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b154f-f491-418e-86cb-6d4007fe707f None admin] RBAC: Adding query filter params (type=network) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:37.788481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b154f-f491-418e-86cb-6d4007fe707f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.789854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b154f-f491-418e-86cb-6d4007fe707f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:37.794833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 408/818] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:05:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:37.795694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b154f-f491-418e-86cb-6d4007fe707f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.809851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 411/819] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:05:37 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/12eafcbf5c98425782138d5363db9a25/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:05:37.811150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93378b83-3159-4771-9903-7d1d6ff7db82 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:37.839281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:37.940914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93378b83-3159-4771-9903-7d1d6ff7db82 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.967068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b154f-f491-418e-86cb-6d4007fe707f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:37.990074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e402d-fffb-4a9f-abd1-5e76270e240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:37.990622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e402d-fffb-4a9f-abd1-5e76270e240b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:37.990977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e402d-fffb-4a9f-abd1-5e76270e240b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:37.991190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e402d-fffb-4a9f-abd1-5e76270e240b None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:37.992298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e402d-fffb-4a9f-abd1-5e76270e240b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:37.992298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e402d-fffb-4a9f-abd1-5e76270e240b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:37.993690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e402d-fffb-4a9f-abd1-5e76270e240b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:38.000268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 412/820] 10.222.0.83 () {56 vars in 1124 bytes} [Wed Aug 24 11:05:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:38.000976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e402d-fffb-4a9f-abd1-5e76270e240b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:38.021318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eb3a5-dda9-4098-8fb3-07a7bfcedf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:38.021986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eb3a5-dda9-4098-8fb3-07a7bfcedf9f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:38.022272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eb3a5-dda9-4098-8fb3-07a7bfcedf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:38.022549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eb3a5-dda9-4098-8fb3-07a7bfcedf9f None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:38.023727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1eb3a5-dda9-4098-8fb3-07a7bfcedf9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:38.023727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1eb3a5-dda9-4098-8fb3-07a7bfcedf9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:38.024914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1eb3a5-dda9-4098-8fb3-07a7bfcedf9f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:38.040698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 409/821] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:38.042072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eb3a5-dda9-4098-8fb3-07a7bfcedf9f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:38.049164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:38.159312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:38.167358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eb3a5-dda9-4098-8fb3-07a7bfcedf9f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:38.214907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f24b5-7fd4-48da-b0b9-f26fd31f0afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:38.215528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f24b5-7fd4-48da-b0b9-f26fd31f0afb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:38.215813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f24b5-7fd4-48da-b0b9-f26fd31f0afb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:38.216171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f24b5-7fd4-48da-b0b9-f26fd31f0afb None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:38.216857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f24b5-7fd4-48da-b0b9-f26fd31f0afb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:38.217168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f24b5-7fd4-48da-b0b9-f26fd31f0afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:38.220640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f24b5-7fd4-48da-b0b9-f26fd31f0afb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:38.231279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 410/822] 10.222.0.83 () {56 vars in 1124 bytes} [Wed Aug 24 11:05:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:38.231279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f24b5-7fd4-48da-b0b9-f26fd31f0afb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:38.253641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e402d-fffb-4a9f-abd1-5e76270e240b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:38.308138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff72df-066c-4373-b722-a5299d785acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:38.308757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff72df-066c-4373-b722-a5299d785acf None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:38.309040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff72df-066c-4373-b722-a5299d785acf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:38.309383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff72df-066c-4373-b722-a5299d785acf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:38.310101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff72df-066c-4373-b722-a5299d785acf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:38.312377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff72df-066c-4373-b722-a5299d785acf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:38.355658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f24b5-7fd4-48da-b0b9-f26fd31f0afb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:38.394547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63421-e98e-4cd2-a53d-e7e6f8b3a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:38.395170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63421-e98e-4cd2-a53d-e7e6f8b3a773 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:38.395444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63421-e98e-4cd2-a53d-e7e6f8b3a773 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:38.395714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63421-e98e-4cd2-a53d-e7e6f8b3a773 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:38.403652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63421-e98e-4cd2-a53d-e7e6f8b3a773 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:38.405109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63421-e98e-4cd2-a53d-e7e6f8b3a773 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-1ec63421-e98e-4cd2-a53d-e7e6f8b3a773 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:38.418018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:38.420945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 411/823] 10.222.0.83 () {56 vars in 1093 bytes} [Wed Aug 24 11:05:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:38.421984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63421-e98e-4cd2-a53d-e7e6f8b3a773 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:38.444282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:38.466614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 413/824] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:05:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:38.467964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff72df-066c-4373-b722-a5299d785acf None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:38.645221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63421-e98e-4cd2-a53d-e7e6f8b3a773 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:38.658623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:38.684372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff72df-066c-4373-b722-a5299d785acf None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:38.691129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c6879-bf84-45a3-9081-a9233e323813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:38.692084 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c6879-bf84-45a3-9081-a9233e323813 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:38.692572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c6879-bf84-45a3-9081-a9233e323813 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:38.693050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c6879-bf84-45a3-9081-a9233e323813 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:38.693984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c6879-bf84-45a3-9081-a9233e323813 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:38.695737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c6879-bf84-45a3-9081-a9233e323813 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:38.723803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ae17d-2842-4b89-bbe6-72a259409979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:38.724997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ae17d-2842-4b89-bbe6-72a259409979 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:38.724997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ae17d-2842-4b89-bbe6-72a259409979 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:38.724997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ae17d-2842-4b89-bbe6-72a259409979 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:38.725695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ae17d-2842-4b89-bbe6-72a259409979 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:38.727035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ae17d-2842-4b89-bbe6-72a259409979 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:38.728529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ae17d-2842-4b89-bbe6-72a259409979 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:38.752508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 414/825] 10.222.0.83 () {56 vars in 1133 bytes} [Wed Aug 24 11:05:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:38.759037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ae17d-2842-4b89-bbe6-72a259409979 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:38.794826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 412/826] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:05:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:38.869344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fddba5-6ceb-4a83-93b4-2ca82111a6f1 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:38.870091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fddba5-6ceb-4a83-93b4-2ca82111a6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:38.870599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fddba5-6ceb-4a83-93b4-2ca82111a6f1 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:38.872798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 413/827] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:38.880044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c10d0-cec3-4df8-a337-1be4cd8a3c57 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:38.880815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c10d0-cec3-4df8-a337-1be4cd8a3c57 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:38.881342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c10d0-cec3-4df8-a337-1be4cd8a3c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:38.910238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cf5c10d0-cec3-4df8-a337-1be4cd8a3c57 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:38.912822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cf5c10d0-cec3-4df8-a337-1be4cd8a3c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:38.927737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ae17d-2842-4b89-bbe6-72a259409979 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:38.998616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1bd28-3b08-4641-a2a7-f57995186b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:38.999252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1bd28-3b08-4641-a2a7-f57995186b70 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:38.999573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1bd28-3b08-4641-a2a7-f57995186b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:39.000381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1bd28-3b08-4641-a2a7-f57995186b70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:39.002337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:39.007095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:39.013897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 414/828] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:05:39.090366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1bd28-3b08-4641-a2a7-f57995186b70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:39.091882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1bd28-3b08-4641-a2a7-f57995186b70 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:39.216736 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 415/829] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:05:39.218722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c10d0-cec3-4df8-a337-1be4cd8a3c57 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:39.332013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1bd28-3b08-4641-a2a7-f57995186b70 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:39.363012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c10d0-cec3-4df8-a337-1be4cd8a3c57 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:39.403981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1938083-9779-4c8d-85e5-590be2d85219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:39.404619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1938083-9779-4c8d-85e5-590be2d85219 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:39.404896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1938083-9779-4c8d-85e5-590be2d85219 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:39.405225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1938083-9779-4c8d-85e5-590be2d85219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:39.417442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1938083-9779-4c8d-85e5-590be2d85219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:39.419106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1938083-9779-4c8d-85e5-590be2d85219 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:39.434539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b1938083-9779-4c8d-85e5-590be2d85219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:39.436967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:39.439504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:39.441780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:39.441780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:39.441780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:39.442361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 415/830] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:39.443770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1938083-9779-4c8d-85e5-590be2d85219 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:39.500492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:39.554378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1938083-9779-4c8d-85e5-590be2d85219 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:39.594102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48307d-881e-47f6-9958-5ed4be1de19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:39.594721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48307d-881e-47f6-9958-5ed4be1de19a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:39.595036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48307d-881e-47f6-9958-5ed4be1de19a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:39.595345 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48307d-881e-47f6-9958-5ed4be1de19a None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:39.595984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48307d-881e-47f6-9958-5ed4be1de19a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:39.596288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48307d-881e-47f6-9958-5ed4be1de19a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:39.597721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48307d-881e-47f6-9958-5ed4be1de19a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:39.615818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 416/831] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:39.616812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48307d-881e-47f6-9958-5ed4be1de19a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:39.618347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1bd28-3b08-4641-a2a7-f57995186b70 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:39.708900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f8f7c-8dcc-4abc-a1b7-024c37789097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:39.710707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f8f7c-8dcc-4abc-a1b7-024c37789097 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:39.711708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f8f7c-8dcc-4abc-a1b7-024c37789097 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:39.712930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f8f7c-8dcc-4abc-a1b7-024c37789097 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:39.728174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48307d-881e-47f6-9958-5ed4be1de19a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:39.741036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f8f7c-8dcc-4abc-a1b7-024c37789097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:39.743013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d96f8f7c-8dcc-4abc-a1b7-024c37789097 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:39.766439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:39.768510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 416/832] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:05:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:39.770343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f8f7c-8dcc-4abc-a1b7-024c37789097 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:39.775110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e339-5ba9-43c9-87df-655b6b0c457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:39.775110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e339-5ba9-43c9-87df-655b6b0c457f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:39.775110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e339-5ba9-43c9-87df-655b6b0c457f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:39.775110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e339-5ba9-43c9-87df-655b6b0c457f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:39.779785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88e339-5ba9-43c9-87df-655b6b0c457f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:39.783159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88e339-5ba9-43c9-87df-655b6b0c457f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4a88e339-5ba9-43c9-87df-655b6b0c457f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:39.797667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:39.799914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:39.802090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:39.802090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:39.802090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:39.802090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:39.802090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:39.802090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:39.802090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:39.802090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:39.810337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 417/833] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:39.811408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e339-5ba9-43c9-87df-655b6b0c457f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:39.934957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e339-5ba9-43c9-87df-655b6b0c457f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:39.974194 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49121677-4683-4ba6-bb49-d4998dedeedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:39.974935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49121677-4683-4ba6-bb49-d4998dedeedf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:39.975377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49121677-4683-4ba6-bb49-d4998dedeedf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:39.975580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49121677-4683-4ba6-bb49-d4998dedeedf None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:39.976238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49121677-4683-4ba6-bb49-d4998dedeedf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:39.976646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49121677-4683-4ba6-bb49-d4998dedeedf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:39.977999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49121677-4683-4ba6-bb49-d4998dedeedf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:39.994326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 418/834] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:39.994326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cf6ea-fda4-4ebe-b15f-1eac487e2229 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:39.994326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cf6ea-fda4-4ebe-b15f-1eac487e2229 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:39.994326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cf6ea-fda4-4ebe-b15f-1eac487e2229 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:39.994719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 419/835] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:39.996888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cf6ea-fda4-4ebe-b15f-1eac487e2229 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:40.026580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:40.032660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f8f7c-8dcc-4abc-a1b7-024c37789097 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:40.042462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:40.071975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd9f8b-4b73-4aa0-9b84-8204360c81eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:40.072588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9f8b-4b73-4aa0-9b84-8204360c81eb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:40.072866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9f8b-4b73-4aa0-9b84-8204360c81eb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:40.073140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9f8b-4b73-4aa0-9b84-8204360c81eb None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:40.073820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd9f8b-4b73-4aa0-9b84-8204360c81eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:40.074189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd9f8b-4b73-4aa0-9b84-8204360c81eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:40.075668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd9f8b-4b73-4aa0-9b84-8204360c81eb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:40.085678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 417/836] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:05:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:40.091275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640aa9c-6e13-405e-86de-7aa679e0f7fb None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:40.091724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640aa9c-6e13-405e-86de-7aa679e0f7fb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:40.091925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640aa9c-6e13-405e-86de-7aa679e0f7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:40.112583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8640aa9c-6e13-405e-86de-7aa679e0f7fb None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:40.114817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8640aa9c-6e13-405e-86de-7aa679e0f7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:40.162898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cf6ea-fda4-4ebe-b15f-1eac487e2229 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:40.208012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 418/837] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:40.208669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640aa9c-6e13-405e-86de-7aa679e0f7fb None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:40.231251 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6d6f0-fa28-4e52-80e4-06885ada7f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:40.232362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6d6f0-fa28-4e52-80e4-06885ada7f4c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:40.232860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6d6f0-fa28-4e52-80e4-06885ada7f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:40.233610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6d6f0-fa28-4e52-80e4-06885ada7f4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:40.235070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6d6f0-fa28-4e52-80e4-06885ada7f4c None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=32a28f9fb3134f33819e53e0c72c63b4) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:40.235598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6d6f0-fa28-4e52-80e4-06885ada7f4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:40.237252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6d6f0-fa28-4e52-80e4-06885ada7f4c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:40.246842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 420/838] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:39 2022] GET /identity/v3/role_assignments?user.id=32a28f9fb3134f33819e53e0c72c63b4&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:40.320524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:40.336031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640aa9c-6e13-405e-86de-7aa679e0f7fb None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:40.375739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4a849-0a8f-4eff-8d2f-d577d4b671f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:40.376580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4a849-0a8f-4eff-8d2f-d577d4b671f0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:40.376952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4a849-0a8f-4eff-8d2f-d577d4b671f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:40.377318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4a849-0a8f-4eff-8d2f-d577d4b671f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:40.388970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4a849-0a8f-4eff-8d2f-d577d4b671f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:40.391195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4a849-0a8f-4eff-8d2f-d577d4b671f0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-bbb4a849-0a8f-4eff-8d2f-d577d4b671f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:40.408436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:40.409648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:40.410801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:40.410801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:40.410801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:40.413940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 419/839] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:40.417682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6d6f0-fa28-4e52-80e4-06885ada7f4c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:40.491251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4a849-0a8f-4eff-8d2f-d577d4b671f0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:40.627438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4a849-0a8f-4eff-8d2f-d577d4b671f0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:40.649842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:40.657400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6d6f0-fa28-4e52-80e4-06885ada7f4c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:40.681188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9e0f7-167c-49bd-8ad6-3b0c0ab89541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:40.682964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9e0f7-167c-49bd-8ad6-3b0c0ab89541 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:40.683545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9e0f7-167c-49bd-8ad6-3b0c0ab89541 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:40.684108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9e0f7-167c-49bd-8ad6-3b0c0ab89541 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:40.696970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:40.698901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88fa1c-df63-4efd-a610-d5149d2d5103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:40.699756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88fa1c-df63-4efd-a610-d5149d2d5103 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:40.700128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88fa1c-df63-4efd-a610-d5149d2d5103 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:40.700492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88fa1c-df63-4efd-a610-d5149d2d5103 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:40.701339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88fa1c-df63-4efd-a610-d5149d2d5103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:40.701797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88fa1c-df63-4efd-a610-d5149d2d5103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:40.702969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9e0f7-167c-49bd-8ad6-3b0c0ab89541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:40.703449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88fa1c-df63-4efd-a610-d5149d2d5103 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-13a9e0f7-167c-49bd-8ad6-3b0c0ab89541 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:40.709243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:40.712667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:05:40.713696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:40.714645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 420/840] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:40.717920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9e0f7-167c-49bd-8ad6-3b0c0ab89541 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:40.724387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 421/841] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:40.725756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88fa1c-df63-4efd-a610-d5149d2d5103 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:40.850955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88fa1c-df63-4efd-a610-d5149d2d5103 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:40.855586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9e0f7-167c-49bd-8ad6-3b0c0ab89541 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:40.895354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd84fec-d3ad-4d7d-9170-db1ccd36145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:40.896241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd84fec-d3ad-4d7d-9170-db1ccd36145d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:40.896627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd84fec-d3ad-4d7d-9170-db1ccd36145d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:40.896999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd84fec-d3ad-4d7d-9170-db1ccd36145d None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:40.897791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd84fec-d3ad-4d7d-9170-db1ccd36145d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:40.898222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd84fec-d3ad-4d7d-9170-db1ccd36145d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:40.899731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd84fec-d3ad-4d7d-9170-db1ccd36145d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:40.900839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef844c9d-46ce-48a7-ba66-97f4c5b94daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:40.901998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef844c9d-46ce-48a7-ba66-97f4c5b94daa None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:40.902895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef844c9d-46ce-48a7-ba66-97f4c5b94daa None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:40.903327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef844c9d-46ce-48a7-ba66-97f4c5b94daa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:40.913130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef844c9d-46ce-48a7-ba66-97f4c5b94daa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:40.914791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 421/842] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:40.915376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef844c9d-46ce-48a7-ba66-97f4c5b94daa None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:40.916072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd84fec-d3ad-4d7d-9170-db1ccd36145d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-ef844c9d-46ce-48a7-ba66-97f4c5b94daa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:40.933131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:40.934416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:40.935996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:40.939497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 422/843] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:05:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:40.944879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef844c9d-46ce-48a7-ba66-97f4c5b94daa None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:40.975259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:40.999159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:41.045501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd84fec-d3ad-4d7d-9170-db1ccd36145d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:41.086105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49209689-d10f-4a69-a778-c633931419c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:41.087034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49209689-d10f-4a69-a778-c633931419c2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.087468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49209689-d10f-4a69-a778-c633931419c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.087965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49209689-d10f-4a69-a778-c633931419c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.100390 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49209689-d10f-4a69-a778-c633931419c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:41.101991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49209689-d10f-4a69-a778-c633931419c2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:41.112482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef844c9d-46ce-48a7-ba66-97f4c5b94daa None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-49209689-d10f-4a69-a778-c633931419c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:41.118078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:41.124909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:41.125921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:41.125921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:41.125921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:41.125921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 422/844] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:41.125921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5fbc9-21fc-4474-b76c-e32bf73e2484 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.125921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5fbc9-21fc-4474-b76c-e32bf73e2484 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.125921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5fbc9-21fc-4474-b76c-e32bf73e2484 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.125921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 423/845] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:41.125921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5fbc9-21fc-4474-b76c-e32bf73e2484 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:41.283495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a4af0-bd61-46ac-9468-ff0e57de6262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:41.283495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a4af0-bd61-46ac-9468-ff0e57de6262 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.283495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a4af0-bd61-46ac-9468-ff0e57de6262 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.283495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a4af0-bd61-46ac-9468-ff0e57de6262 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.284031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a4af0-bd61-46ac-9468-ff0e57de6262 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:41.284031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a4af0-bd61-46ac-9468-ff0e57de6262 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:41.284031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a4af0-bd61-46ac-9468-ff0e57de6262 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:41.284031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 423/846] 10.222.0.83 () {56 vars in 1149 bytes} [Wed Aug 24 11:05:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:41.284031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dd47c-9d3c-447b-bed9-63ff303a71af None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.284031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dd47c-9d3c-447b-bed9-63ff303a71af None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.284557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dd47c-9d3c-447b-bed9-63ff303a71af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.284635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5fbc9-21fc-4474-b76c-e32bf73e2484 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:41.284635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:41.284793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a14dd47c-9d3c-447b-bed9-63ff303a71af None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:41.284793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a14dd47c-9d3c-447b-bed9-63ff303a71af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:41.298957 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6249d-66de-4485-9aad-d62c754e7dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:41.299770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6249d-66de-4485-9aad-d62c754e7dfb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.299932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6249d-66de-4485-9aad-d62c754e7dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.300245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6249d-66de-4485-9aad-d62c754e7dfb None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.300969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6249d-66de-4485-9aad-d62c754e7dfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:41.301276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6249d-66de-4485-9aad-d62c754e7dfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:41.302738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6249d-66de-4485-9aad-d62c754e7dfb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:41.321337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 424/847] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:41.321337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6249d-66de-4485-9aad-d62c754e7dfb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:41.359166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 424/848] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:41.360032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dd47c-9d3c-447b-bed9-63ff303a71af None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:41.374774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:41.447214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6249d-66de-4485-9aad-d62c754e7dfb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:41.486886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dd47c-9d3c-447b-bed9-63ff303a71af None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:41.487112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf17d1-fcd3-4a55-a758-d6cadf4af1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:41.487543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf17d1-fcd3-4a55-a758-d6cadf4af1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.487826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf17d1-fcd3-4a55-a758-d6cadf4af1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.488104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf17d1-fcd3-4a55-a758-d6cadf4af1e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.524846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669f120-06e2-4776-a524-f845d2aa2183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:41.526079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669f120-06e2-4776-a524-f845d2aa2183 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.527356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669f120-06e2-4776-a524-f845d2aa2183 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.527860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669f120-06e2-4776-a524-f845d2aa2183 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.536155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:41.538353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669f120-06e2-4776-a524-f845d2aa2183 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:41.540792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669f120-06e2-4776-a524-f845d2aa2183 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-5669f120-06e2-4776-a524-f845d2aa2183 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:41.558076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:41.559127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:05:41.560060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:41.562847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 425/849] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:05:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:41.571440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669f120-06e2-4776-a524-f845d2aa2183 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:41.574362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf17d1-fcd3-4a55-a758-d6cadf4af1e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:41.576499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf17d1-fcd3-4a55-a758-d6cadf4af1e0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:41.659862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:41.710065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 425/850] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:05:41.710965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf17d1-fcd3-4a55-a758-d6cadf4af1e0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:41.717114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669f120-06e2-4776-a524-f845d2aa2183 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:41.760407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2271dc6-c778-454b-80a7-151502ffa653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:41.761050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2271dc6-c778-454b-80a7-151502ffa653 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.761323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2271dc6-c778-454b-80a7-151502ffa653 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.761570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2271dc6-c778-454b-80a7-151502ffa653 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.762346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2271dc6-c778-454b-80a7-151502ffa653 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:41.762674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2271dc6-c778-454b-80a7-151502ffa653 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:41.764382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2271dc6-c778-454b-80a7-151502ffa653 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:41.785659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 426/851] 10.222.0.83 () {56 vars in 1135 bytes} [Wed Aug 24 11:05:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:41.786625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2271dc6-c778-454b-80a7-151502ffa653 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:41.801413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:41.829121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf17d1-fcd3-4a55-a758-d6cadf4af1e0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:41.867936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff25557-9bbf-4509-b327-b43a4db8f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:41.868682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff25557-9bbf-4509-b327-b43a4db8f250 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.868915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff25557-9bbf-4509-b327-b43a4db8f250 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.869119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff25557-9bbf-4509-b327-b43a4db8f250 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.870109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff25557-9bbf-4509-b327-b43a4db8f250 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:41.871827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff25557-9bbf-4509-b327-b43a4db8f250 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-9ff25557-9bbf-4509-b327-b43a4db8f250 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:41.881414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:41.882914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:41.883782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 426/852] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:05:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:41.884629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff25557-9bbf-4509-b327-b43a4db8f250 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:41.897533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2271dc6-c778-454b-80a7-151502ffa653 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:41.923771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:41.938608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674fa9a-c2a6-4855-b57b-e9fc0ed32af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:41.940737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674fa9a-c2a6-4855-b57b-e9fc0ed32af4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:41.940737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674fa9a-c2a6-4855-b57b-e9fc0ed32af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:41.940737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674fa9a-c2a6-4855-b57b-e9fc0ed32af4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:41.951296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674fa9a-c2a6-4855-b57b-e9fc0ed32af4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:41.953316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674fa9a-c2a6-4855-b57b-e9fc0ed32af4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8674fa9a-c2a6-4855-b57b-e9fc0ed32af4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:41.969412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:41.970590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:41.971525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:41.971525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:41.971525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:41.971525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 427/853] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:41.972193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674fa9a-c2a6-4855-b57b-e9fc0ed32af4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:42.007894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff25557-9bbf-4509-b327-b43a4db8f250 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:42.044280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e036538-94eb-4bc1-b611-33fd1a0553b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:42.044813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e036538-94eb-4bc1-b611-33fd1a0553b5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.045097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e036538-94eb-4bc1-b611-33fd1a0553b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.045416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e036538-94eb-4bc1-b611-33fd1a0553b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.055402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:42.089664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674fa9a-c2a6-4855-b57b-e9fc0ed32af4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:42.123083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e036538-94eb-4bc1-b611-33fd1a0553b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:42.124609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e036538-94eb-4bc1-b611-33fd1a0553b5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:42.126463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798c856-79f0-4d39-8816-60277ceea2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:42.127124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798c856-79f0-4d39-8816-60277ceea2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.127398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798c856-79f0-4d39-8816-60277ceea2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.127634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798c856-79f0-4d39-8816-60277ceea2d0 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.128371 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798c856-79f0-4d39-8816-60277ceea2d0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:42.128645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798c856-79f0-4d39-8816-60277ceea2d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:42.130024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798c856-79f0-4d39-8816-60277ceea2d0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:42.138000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 428/854] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:05:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:42.138000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798c856-79f0-4d39-8816-60277ceea2d0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:42.183763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:42.248093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 427/855] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:05:42.249129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e036538-94eb-4bc1-b611-33fd1a0553b5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:42.256901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798c856-79f0-4d39-8816-60277ceea2d0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:42.298198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:42.298754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347d466-e997-4ce1-ae06-c118bce5a304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:42.299380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347d466-e997-4ce1-ae06-c118bce5a304 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.299651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347d466-e997-4ce1-ae06-c118bce5a304 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.300298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347d466-e997-4ce1-ae06-c118bce5a304 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.300988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347d466-e997-4ce1-ae06-c118bce5a304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:42.301331 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347d466-e997-4ce1-ae06-c118bce5a304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:42.303097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347d466-e997-4ce1-ae06-c118bce5a304 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:42.316945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 429/856] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:42.318068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347d466-e997-4ce1-ae06-c118bce5a304 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:42.372847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e036538-94eb-4bc1-b611-33fd1a0553b5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:42.409391 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a36c0-fc6b-4109-8895-53d2f0f946e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:42.410034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a36c0-fc6b-4109-8895-53d2f0f946e3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.410329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a36c0-fc6b-4109-8895-53d2f0f946e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.410618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a36c0-fc6b-4109-8895-53d2f0f946e3 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.411253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a36c0-fc6b-4109-8895-53d2f0f946e3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:42.411563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a36c0-fc6b-4109-8895-53d2f0f946e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:42.413018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a36c0-fc6b-4109-8895-53d2f0f946e3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:42.417679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 428/857] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:05:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:42.420437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a36c0-fc6b-4109-8895-53d2f0f946e3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:42.472489 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:42.480875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347d466-e997-4ce1-ae06-c118bce5a304 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:42.546990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023203a0-dcf8-4ab7-afff-0cb85b9f81aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:42.547779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023203a0-dcf8-4ab7-afff-0cb85b9f81aa None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.548129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023203a0-dcf8-4ab7-afff-0cb85b9f81aa None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.548609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023203a0-dcf8-4ab7-afff-0cb85b9f81aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.563499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023203a0-dcf8-4ab7-afff-0cb85b9f81aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:42.566183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023203a0-dcf8-4ab7-afff-0cb85b9f81aa None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:42.567033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a36c0-fc6b-4109-8895-53d2f0f946e3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:42.569138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-023203a0-dcf8-4ab7-afff-0cb85b9f81aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:42.592976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:42.594034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:42.594936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:42.594936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:42.594936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:42.594936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 430/858] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:42.596592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206d868-8fd7-46d0-aef9-158ec678f52a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.597114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206d868-8fd7-46d0-aef9-158ec678f52a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.597529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206d868-8fd7-46d0-aef9-158ec678f52a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.601933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 431/859] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:42.605277 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98d022-cb72-439e-a9f3-b638969e25d9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.605600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98d022-cb72-439e-a9f3-b638969e25d9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.605832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98d022-cb72-439e-a9f3-b638969e25d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.608298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5e7fd-a772-4d6f-927a-82a91ad48867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:42.609269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5e7fd-a772-4d6f-927a-82a91ad48867 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.609512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5e7fd-a772-4d6f-927a-82a91ad48867 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.609792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5e7fd-a772-4d6f-927a-82a91ad48867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.619315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5e7fd-a772-4d6f-927a-82a91ad48867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:42.620924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5e7fd-a772-4d6f-927a-82a91ad48867 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:42.630017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3f98d022-cb72-439e-a9f3-b638969e25d9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:42.631740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3f98d022-cb72-439e-a9f3-b638969e25d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-16a5e7fd-a772-4d6f-927a-82a91ad48867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:42.635392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:42.636301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:42.637212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:42.637212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:42.637212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:42.637212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:42.637212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:42.637212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:42.637212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:42.637212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:42.638323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 429/860] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:42.642094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5e7fd-a772-4d6f-927a-82a91ad48867 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:42.755690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5e7fd-a772-4d6f-927a-82a91ad48867 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:42.755894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 432/861] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:05:42.756603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98d022-cb72-439e-a9f3-b638969e25d9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:42.814081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1cceb-8d9c-46e7-a92b-276e29d8ce3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:42.815039 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1cceb-8d9c-46e7-a92b-276e29d8ce3b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.815536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1cceb-8d9c-46e7-a92b-276e29d8ce3b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.816526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1cceb-8d9c-46e7-a92b-276e29d8ce3b None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.817668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1cceb-8d9c-46e7-a92b-276e29d8ce3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:42.818198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1cceb-8d9c-46e7-a92b-276e29d8ce3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:42.820483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1cceb-8d9c-46e7-a92b-276e29d8ce3b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:42.842603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 430/862] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:42.844363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1cceb-8d9c-46e7-a92b-276e29d8ce3b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:42.849604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:42.871324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98d022-cb72-439e-a9f3-b638969e25d9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:42.878765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:42.912988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eac49-f8a5-4655-8e4f-44da78dfab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:42.913895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eac49-f8a5-4655-8e4f-44da78dfab8e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:42.914354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eac49-f8a5-4655-8e4f-44da78dfab8e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:42.914727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eac49-f8a5-4655-8e4f-44da78dfab8e None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:42.915766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eac49-f8a5-4655-8e4f-44da78dfab8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:42.916243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eac49-f8a5-4655-8e4f-44da78dfab8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:42.918021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eac49-f8a5-4655-8e4f-44da78dfab8e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:42.939924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 433/863] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:42.941415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eac49-f8a5-4655-8e4f-44da78dfab8e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:42.993208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1cceb-8d9c-46e7-a92b-276e29d8ce3b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:43.044223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66a2ab-b894-42b6-9488-9905ba453309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:43.044858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66a2ab-b894-42b6-9488-9905ba453309 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:43.045140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66a2ab-b894-42b6-9488-9905ba453309 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:43.045424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66a2ab-b894-42b6-9488-9905ba453309 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:43.053478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66a2ab-b894-42b6-9488-9905ba453309 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:43.055089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66a2ab-b894-42b6-9488-9905ba453309 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3d66a2ab-b894-42b6-9488-9905ba453309 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:43.076085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:43.077100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 431/864] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:43.078976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66a2ab-b894-42b6-9488-9905ba453309 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:43.096263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eac49-f8a5-4655-8e4f-44da78dfab8e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:43.140329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ab070-3298-4c44-bdfb-98b876c7d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:43.141844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ab070-3298-4c44-bdfb-98b876c7d561 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:43.142345 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ab070-3298-4c44-bdfb-98b876c7d561 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:43.142921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ab070-3298-4c44-bdfb-98b876c7d561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:43.144115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ab070-3298-4c44-bdfb-98b876c7d561 None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=f9d0b6662bf947cba853388819a933b8) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:43.144835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ab070-3298-4c44-bdfb-98b876c7d561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:43.146633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ab070-3298-4c44-bdfb-98b876c7d561 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:43.161916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 434/865] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:42 2022] GET /identity/v3/role_assignments?user.id=f9d0b6662bf947cba853388819a933b8&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:43.161916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ab070-3298-4c44-bdfb-98b876c7d561 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:43.168172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:43.176968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:43.208572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66a2ab-b894-42b6-9488-9905ba453309 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:43.272874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5552ef-f30f-4c49-ab03-a4c592d3707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:43.272874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5552ef-f30f-4c49-ab03-a4c592d3707d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:43.272874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5552ef-f30f-4c49-ab03-a4c592d3707d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:43.272874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5552ef-f30f-4c49-ab03-a4c592d3707d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:43.273313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5552ef-f30f-4c49-ab03-a4c592d3707d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:43.273313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5552ef-f30f-4c49-ab03-a4c592d3707d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-5a5552ef-f30f-4c49-ab03-a4c592d3707d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:43.286278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:43.287547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:05:43.288582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:43.296655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 432/866] 10.222.0.83 () {56 vars in 1132 bytes} [Wed Aug 24 11:05:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:43.297856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5552ef-f30f-4c49-ab03-a4c592d3707d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:43.333410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ab070-3298-4c44-bdfb-98b876c7d561 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:43.380960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cb676-7357-4dfd-b473-870609429567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:43.381949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cb676-7357-4dfd-b473-870609429567 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:43.382462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cb676-7357-4dfd-b473-870609429567 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:43.382914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cb676-7357-4dfd-b473-870609429567 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:43.383765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cb676-7357-4dfd-b473-870609429567 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:43.384226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cb676-7357-4dfd-b473-870609429567 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:43.385817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cb676-7357-4dfd-b473-870609429567 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:43.406128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 435/867] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:43.410663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cb676-7357-4dfd-b473-870609429567 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:43.456222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5552ef-f30f-4c49-ab03-a4c592d3707d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:43.496315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:43.502959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fd552-a945-45e0-adfe-58b2b90781b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:43.504157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fd552-a945-45e0-adfe-58b2b90781b9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:43.504605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fd552-a945-45e0-adfe-58b2b90781b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:43.505027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fd552-a945-45e0-adfe-58b2b90781b9 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:43.506016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fd552-a945-45e0-adfe-58b2b90781b9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:43.507305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fd552-a945-45e0-adfe-58b2b90781b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:43.509527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fd552-a945-45e0-adfe-58b2b90781b9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:43.515842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 433/868] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:43.522965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fd552-a945-45e0-adfe-58b2b90781b9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:43.537581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:43.542882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cb676-7357-4dfd-b473-870609429567 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:43.584376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921f45b-d640-496c-89ff-adc6e88adc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:43.585107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f45b-d640-496c-89ff-adc6e88adc71 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:43.585485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f45b-d640-496c-89ff-adc6e88adc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:43.585924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f45b-d640-496c-89ff-adc6e88adc71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:43.586799 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f45b-d640-496c-89ff-adc6e88adc71 None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=12eafcbf5c98425782138d5363db9a25) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:43.587215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f45b-d640-496c-89ff-adc6e88adc71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:43.589528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f45b-d640-496c-89ff-adc6e88adc71 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:43.605909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 436/869] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:43 2022] GET /identity/v3/role_assignments?user.id=12eafcbf5c98425782138d5363db9a25&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:43.653620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fd552-a945-45e0-adfe-58b2b90781b9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:43.700328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b9293-92fc-4bfb-a7fe-0c8c4c6b5448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:43.701037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b9293-92fc-4bfb-a7fe-0c8c4c6b5448 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:43.701326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b9293-92fc-4bfb-a7fe-0c8c4c6b5448 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:43.701605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b9293-92fc-4bfb-a7fe-0c8c4c6b5448 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:43.702337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b9293-92fc-4bfb-a7fe-0c8c4c6b5448 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:43.702652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b9293-92fc-4bfb-a7fe-0c8c4c6b5448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:43.704635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b9293-92fc-4bfb-a7fe-0c8c4c6b5448 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:43.710589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 434/870] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:44.490869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1468ab-9ff9-4976-b083-dbbcf91a933a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:44.491613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1468ab-9ff9-4976-b083-dbbcf91a933a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:44.491613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1468ab-9ff9-4976-b083-dbbcf91a933a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:44.493081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 437/871] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:44.500294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42408-166f-43a3-8cb6-ebc07d642487 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:44.500792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42408-166f-43a3-8cb6-ebc07d642487 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:44.500878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42408-166f-43a3-8cb6-ebc07d642487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:44.521394 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4cf42408-166f-43a3-8cb6-ebc07d642487 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:44.522649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4cf42408-166f-43a3-8cb6-ebc07d642487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:44.622086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 435/872] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:44.892351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1468ab-9ff9-4976-b083-dbbcf91a933a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:45.013144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1468ab-9ff9-4976-b083-dbbcf91a933a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:45.043882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:45.052594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39f4b0-0a76-4a4c-b75b-5e79319cc4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:45.053338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39f4b0-0a76-4a4c-b75b-5e79319cc4ef None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:45.053628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39f4b0-0a76-4a4c-b75b-5e79319cc4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:45.053937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39f4b0-0a76-4a4c-b75b-5e79319cc4ef None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:45.054661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39f4b0-0a76-4a4c-b75b-5e79319cc4ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:45.056085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39f4b0-0a76-4a4c-b75b-5e79319cc4ef None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:45.170436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 438/873] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:05:44 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:45.369117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0589202-a140-4cc6-8d61-32143bfa147a None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:45.369739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0589202-a140-4cc6-8d61-32143bfa147a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:45.370268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0589202-a140-4cc6-8d61-32143bfa147a None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:45.372134 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 436/874] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:45.378095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854d480-ff1a-4c4b-99e7-f9dca8d3d446 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:45.378381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854d480-ff1a-4c4b-99e7-f9dca8d3d446 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:45.378658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854d480-ff1a-4c4b-99e7-f9dca8d3d446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:45.397846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0854d480-ff1a-4c4b-99e7-f9dca8d3d446 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:45.399093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0854d480-ff1a-4c4b-99e7-f9dca8d3d446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:45.481932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 439/875] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:45.701962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee9d1-a0fe-4f52-9ff4-02029c498a01 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:45.703016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee9d1-a0fe-4f52-9ff4-02029c498a01 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:45.703118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee9d1-a0fe-4f52-9ff4-02029c498a01 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:45.704634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 437/876] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:45.710601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b910683-38a7-4f75-983b-4f6fe06d2bea None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:45.711051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b910683-38a7-4f75-983b-4f6fe06d2bea None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:45.711125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b910683-38a7-4f75-983b-4f6fe06d2bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:45.730152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4b910683-38a7-4f75-983b-4f6fe06d2bea None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:45.731344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4b910683-38a7-4f75-983b-4f6fe06d2bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:45.735342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af323bd-8fad-4a18-9acf-8902cb800f8e None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:45.735342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af323bd-8fad-4a18-9acf-8902cb800f8e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:45.736314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af323bd-8fad-4a18-9acf-8902cb800f8e None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:45.737316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 438/877] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:45.744803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10010851-ad41-4834-9253-767cb354c2a4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:45.745240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10010851-ad41-4834-9253-767cb354c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:45.745611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10010851-ad41-4834-9253-767cb354c2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:45.755341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:45.766289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-10010851-ad41-4834-9253-767cb354c2a4 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:45.767496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-10010851-ad41-4834-9253-767cb354c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:45.818028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 440/878] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:45.820412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b910683-38a7-4f75-983b-4f6fe06d2bea None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:45.847004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 439/879] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:45.890289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:45.938628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b910683-38a7-4f75-983b-4f6fe06d2bea None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:45.974873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a62f6f-e90a-4546-b9c1-5b8e52dfb1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:45.975759 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a62f6f-e90a-4546-b9c1-5b8e52dfb1ff None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:45.976189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a62f6f-e90a-4546-b9c1-5b8e52dfb1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:45.976611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a62f6f-e90a-4546-b9c1-5b8e52dfb1ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:45.988870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a62f6f-e90a-4546-b9c1-5b8e52dfb1ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-08a62f6f-e90a-4546-b9c1-5b8e52dfb1ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:45.993993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:45.995160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:05:45.996220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:45.998662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 441/880] 10.222.0.83 () {56 vars in 1120 bytes} [Wed Aug 24 11:05:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:46.006673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10010851-ad41-4834-9253-767cb354c2a4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:46.117590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10010851-ad41-4834-9253-767cb354c2a4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:46.127138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a62f6f-e90a-4546-b9c1-5b8e52dfb1ff None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:46.155406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59efc6c-0a2b-4095-89f5-86a774efebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:46.156030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59efc6c-0a2b-4095-89f5-86a774efebb7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:46.156303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59efc6c-0a2b-4095-89f5-86a774efebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:46.156571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59efc6c-0a2b-4095-89f5-86a774efebb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:46.157249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59efc6c-0a2b-4095-89f5-86a774efebb7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:46.157544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59efc6c-0a2b-4095-89f5-86a774efebb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:46.159000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59efc6c-0a2b-4095-89f5-86a774efebb7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:46.160507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:46.167727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 440/881] 10.222.0.83 () {56 vars in 1115 bytes} [Wed Aug 24 11:05:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:46.171863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59efc6c-0a2b-4095-89f5-86a774efebb7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:46.250950 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a62f6f-e90a-4546-b9c1-5b8e52dfb1ff None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:46.254106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:46.282786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59efc6c-0a2b-4095-89f5-86a774efebb7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:46.293839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e16cb-8f6b-435f-8cbc-99fed1bee622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:46.294644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e16cb-8f6b-435f-8cbc-99fed1bee622 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:46.294964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e16cb-8f6b-435f-8cbc-99fed1bee622 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:46.295233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e16cb-8f6b-435f-8cbc-99fed1bee622 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:46.296432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e16cb-8f6b-435f-8cbc-99fed1bee622 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:46.298112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e16cb-8f6b-435f-8cbc-99fed1bee622 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:46.321613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbe23a-5340-4725-accd-4ce35cdef87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:46.322298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbe23a-5340-4725-accd-4ce35cdef87e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:46.322579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbe23a-5340-4725-accd-4ce35cdef87e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:46.322851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbe23a-5340-4725-accd-4ce35cdef87e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:46.323475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbe23a-5340-4725-accd-4ce35cdef87e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:46.324833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbe23a-5340-4725-accd-4ce35cdef87e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6dbbe23a-5340-4725-accd-4ce35cdef87e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:46.334050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:46.335276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:46.336110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 441/882] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:46.336953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbe23a-5340-4725-accd-4ce35cdef87e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:46.403586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:46.422575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 442/883] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:05:46 2022] POST /identity/v3/services => generated 269 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:05:46.423747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e16cb-8f6b-435f-8cbc-99fed1bee622 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:46.467810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbe23a-5340-4725-accd-4ce35cdef87e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:46.532140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f30b9-62e5-4202-a95e-a31567a64d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:46.533148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f30b9-62e5-4202-a95e-a31567a64d2f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:46.534217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f30b9-62e5-4202-a95e-a31567a64d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:46.534563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f30b9-62e5-4202-a95e-a31567a64d2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:46.551848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e16cb-8f6b-435f-8cbc-99fed1bee622 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:46.554728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f30b9-62e5-4202-a95e-a31567a64d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:46.556245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f30b9-62e5-4202-a95e-a31567a64d2f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-325f30b9-62e5-4202-a95e-a31567a64d2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:46.574025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:46.575108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:46.576064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:46.576064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:46.576064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:46.580465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 442/884] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:46.581597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f30b9-62e5-4202-a95e-a31567a64d2f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:46.617096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884acaa-03c6-420d-ac4e-344ea83b8310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:46.618571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884acaa-03c6-420d-ac4e-344ea83b8310 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:46.619076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884acaa-03c6-420d-ac4e-344ea83b8310 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:46.619819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884acaa-03c6-420d-ac4e-344ea83b8310 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:46.620469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884acaa-03c6-420d-ac4e-344ea83b8310 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:46.620770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884acaa-03c6-420d-ac4e-344ea83b8310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:46.622732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884acaa-03c6-420d-ac4e-344ea83b8310 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:46.628257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 443/885] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:05:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:46.635826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884acaa-03c6-420d-ac4e-344ea83b8310 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:46.677723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:46.727938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f30b9-62e5-4202-a95e-a31567a64d2f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:46.739731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:46.758632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884acaa-03c6-420d-ac4e-344ea83b8310 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:46.769633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9648bd-8ba2-4713-9ebe-94e23018d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:46.770713 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9648bd-8ba2-4713-9ebe-94e23018d53b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:46.771143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9648bd-8ba2-4713-9ebe-94e23018d53b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:46.771592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9648bd-8ba2-4713-9ebe-94e23018d53b None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:46.772394 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9648bd-8ba2-4713-9ebe-94e23018d53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:46.772954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9648bd-8ba2-4713-9ebe-94e23018d53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:46.774623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9648bd-8ba2-4713-9ebe-94e23018d53b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:46.794385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 443/886] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:46.796727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9648bd-8ba2-4713-9ebe-94e23018d53b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:46.802742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7f847-e103-4eb5-a057-2f7200674bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:46.803720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f847-e103-4eb5-a057-2f7200674bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:46.804206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f847-e103-4eb5-a057-2f7200674bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:46.804683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f847-e103-4eb5-a057-2f7200674bf8 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:46.805538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f847-e103-4eb5-a057-2f7200674bf8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:46.806100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f847-e103-4eb5-a057-2f7200674bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:46.808630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f847-e103-4eb5-a057-2f7200674bf8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:46.822489 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 444/887] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:05:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:46.927990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9648bd-8ba2-4713-9ebe-94e23018d53b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:46.971320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db861b-4d79-432e-891d-abd54ce875f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:46.972018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db861b-4d79-432e-891d-abd54ce875f8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:46.972301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db861b-4d79-432e-891d-abd54ce875f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:46.972570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db861b-4d79-432e-891d-abd54ce875f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:46.981151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db861b-4d79-432e-891d-abd54ce875f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:46.982663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db861b-4d79-432e-891d-abd54ce875f8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-13db861b-4d79-432e-891d-abd54ce875f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:46.995826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:46.996847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:46.997794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 444/888] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:05:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:47.004871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7f847-e103-4eb5-a057-2f7200674bf8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:47.118320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:47.128423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7f847-e103-4eb5-a057-2f7200674bf8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:47.166430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb93cb-1d68-4f10-a1ee-bc6b6539bd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:47.167047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb93cb-1d68-4f10-a1ee-bc6b6539bd18 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:47.167328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb93cb-1d68-4f10-a1ee-bc6b6539bd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:47.167602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb93cb-1d68-4f10-a1ee-bc6b6539bd18 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:47.168376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb93cb-1d68-4f10-a1ee-bc6b6539bd18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:47.168684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb93cb-1d68-4f10-a1ee-bc6b6539bd18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:47.170393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb93cb-1d68-4f10-a1ee-bc6b6539bd18 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:47.191771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 445/889] 10.222.0.83 () {56 vars in 1149 bytes} [Wed Aug 24 11:05:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:47.197694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db861b-4d79-432e-891d-abd54ce875f8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:47.215423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc275f7-b517-4084-82dc-ab1498457f5c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:47.215763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc275f7-b517-4084-82dc-ab1498457f5c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:47.216243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc275f7-b517-4084-82dc-ab1498457f5c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:47.219521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 446/890] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:47.224599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638ce19-018d-4742-980b-70736caa3a62 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:47.224875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638ce19-018d-4742-980b-70736caa3a62 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:47.225146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638ce19-018d-4742-980b-70736caa3a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:47.238252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:47.248437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1638ce19-018d-4742-980b-70736caa3a62 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:47.250010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1638ce19-018d-4742-980b-70736caa3a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:47.338058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db861b-4d79-432e-891d-abd54ce875f8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:47.354776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 447/891] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:47.390003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20939a0d-34fe-4428-9d3a-8620506d9ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:47.390003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20939a0d-34fe-4428-9d3a-8620506d9ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:47.390003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20939a0d-34fe-4428-9d3a-8620506d9ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:47.390327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20939a0d-34fe-4428-9d3a-8620506d9ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:47.464463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20939a0d-34fe-4428-9d3a-8620506d9ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:47.466113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20939a0d-34fe-4428-9d3a-8620506d9ca6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:47.495347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:47.572976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 445/892] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:47 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:05:47.583005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638ce19-018d-4742-980b-70736caa3a62 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:47.638706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20939a0d-34fe-4428-9d3a-8620506d9ca6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:47.680172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:47.701611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638ce19-018d-4742-980b-70736caa3a62 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:47.753040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2473a64-07b1-41ef-8298-89d176155636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:47.754149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2473a64-07b1-41ef-8298-89d176155636 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:47.754684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2473a64-07b1-41ef-8298-89d176155636 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:47.755206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2473a64-07b1-41ef-8298-89d176155636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:47.768749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2473a64-07b1-41ef-8298-89d176155636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:47.771057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2473a64-07b1-41ef-8298-89d176155636 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:47.776423 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20939a0d-34fe-4428-9d3a-8620506d9ca6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-d2473a64-07b1-41ef-8298-89d176155636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:47.790723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:47.791704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:47.792618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:47.792618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:47.792618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:47.794980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 448/893] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:47.800880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2473a64-07b1-41ef-8298-89d176155636 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:47.809209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:47.818198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c32a6-9241-4b98-a11b-0d84a9847362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:47.824901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c32a6-9241-4b98-a11b-0d84a9847362 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:47.824901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c32a6-9241-4b98-a11b-0d84a9847362 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:47.824901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c32a6-9241-4b98-a11b-0d84a9847362 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:47.824901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c32a6-9241-4b98-a11b-0d84a9847362 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:47.824901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c32a6-9241-4b98-a11b-0d84a9847362 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-653c32a6-9241-4b98-a11b-0d84a9847362 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:47.832575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:47.833794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:05:47.834875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:47.841583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 446/894] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:05:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:47.843687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c32a6-9241-4b98-a11b-0d84a9847362 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:47.941656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2473a64-07b1-41ef-8298-89d176155636 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:47.971752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c32a6-9241-4b98-a11b-0d84a9847362 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:47.982654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:47.989768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31224f9c-c007-4915-9cfa-0b4294984f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:47.990753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31224f9c-c007-4915-9cfa-0b4294984f80 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.000143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31224f9c-c007-4915-9cfa-0b4294984f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.000143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31224f9c-c007-4915-9cfa-0b4294984f80 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.000143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31224f9c-c007-4915-9cfa-0b4294984f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:48.000143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31224f9c-c007-4915-9cfa-0b4294984f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:48.000143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31224f9c-c007-4915-9cfa-0b4294984f80 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:48.010007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049bb49-4279-4d5b-b3f8-b99a70887ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:48.010735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049bb49-4279-4d5b-b3f8-b99a70887ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.010849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 449/895] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:48.011278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049bb49-4279-4d5b-b3f8-b99a70887ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.011527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049bb49-4279-4d5b-b3f8-b99a70887ab4 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.012299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049bb49-4279-4d5b-b3f8-b99a70887ab4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:48.012686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049bb49-4279-4d5b-b3f8-b99a70887ab4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:48.018305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049bb49-4279-4d5b-b3f8-b99a70887ab4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:48.018478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31224f9c-c007-4915-9cfa-0b4294984f80 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:48.028162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 447/896] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:05:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:48.029451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049bb49-4279-4d5b-b3f8-b99a70887ab4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:48.118256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:48.169327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049bb49-4279-4d5b-b3f8-b99a70887ab4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:48.174656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31224f9c-c007-4915-9cfa-0b4294984f80 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:48.209433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac94b08-9615-4b93-8051-2e16418fd950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:48.210032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac94b08-9615-4b93-8051-2e16418fd950 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.210368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac94b08-9615-4b93-8051-2e16418fd950 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.210574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac94b08-9615-4b93-8051-2e16418fd950 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.211276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac94b08-9615-4b93-8051-2e16418fd950 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:48.211724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac94b08-9615-4b93-8051-2e16418fd950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:48.212971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac94b08-9615-4b93-8051-2e16418fd950 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:48.224086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 448/897] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:05:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:48.225043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac94b08-9615-4b93-8051-2e16418fd950 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:48.228096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107ea72-fb76-444d-840d-1c334d2e4146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:48.230093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107ea72-fb76-444d-840d-1c334d2e4146 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.230782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107ea72-fb76-444d-840d-1c334d2e4146 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.231265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107ea72-fb76-444d-840d-1c334d2e4146 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.241093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107ea72-fb76-444d-840d-1c334d2e4146 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:48.243812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107ea72-fb76-444d-840d-1c334d2e4146 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0107ea72-fb76-444d-840d-1c334d2e4146 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:48.261163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:48.262169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:48.263081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:48.263081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:48.263081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:48.263081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:48.263081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:48.263081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:48.263081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:48.263081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:48.263523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 450/898] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:48.270549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107ea72-fb76-444d-840d-1c334d2e4146 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:48.351479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:48.358676 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac94b08-9615-4b93-8051-2e16418fd950 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:48.386876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:48.406496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107ea72-fb76-444d-840d-1c334d2e4146 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:48.407078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7c43c-9958-4b75-ba8e-19b976981aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:48.407813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c43c-9958-4b75-ba8e-19b976981aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.408420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c43c-9958-4b75-ba8e-19b976981aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.408804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c43c-9958-4b75-ba8e-19b976981aa0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.412002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7c43c-9958-4b75-ba8e-19b976981aa0 None admin] RBAC: Adding query filter params (interface=public, service_id=610ef59e5b1948129777e17aa132dc84, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:48.412442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7c43c-9958-4b75-ba8e-19b976981aa0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:48.413982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7c43c-9958-4b75-ba8e-19b976981aa0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:48.422298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 449/899] 10.222.0.83 () {56 vars in 1251 bytes} [Wed Aug 24 11:05:48 2022] GET /identity/v3/endpoints?service_id=610ef59e5b1948129777e17aa132dc84&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:48.429359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7c43c-9958-4b75-ba8e-19b976981aa0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:48.451574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee45f8-b3f6-4ad0-9df6-8454c3e1df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:48.452297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee45f8-b3f6-4ad0-9df6-8454c3e1df60 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.452580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee45f8-b3f6-4ad0-9df6-8454c3e1df60 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.452905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee45f8-b3f6-4ad0-9df6-8454c3e1df60 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.453666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee45f8-b3f6-4ad0-9df6-8454c3e1df60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:48.453967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee45f8-b3f6-4ad0-9df6-8454c3e1df60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:48.455510 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee45f8-b3f6-4ad0-9df6-8454c3e1df60 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:48.474009 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 451/900] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:48.476365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee45f8-b3f6-4ad0-9df6-8454c3e1df60 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:48.563746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7c43c-9958-4b75-ba8e-19b976981aa0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:48.612297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee45f8-b3f6-4ad0-9df6-8454c3e1df60 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:48.617101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7640e1-314d-4748-b2ad-6b5765fa59d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:48.617714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7640e1-314d-4748-b2ad-6b5765fa59d3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.618139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7640e1-314d-4748-b2ad-6b5765fa59d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.618379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7640e1-314d-4748-b2ad-6b5765fa59d3 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.618995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7640e1-314d-4748-b2ad-6b5765fa59d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:48.620402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7640e1-314d-4748-b2ad-6b5765fa59d3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:48.628314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 450/901] 10.222.0.83 () {56 vars in 1087 bytes} [Wed Aug 24 11:05:48 2022] GET /identity/v3/services => generated 1268 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:05:48.629356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501fd76-9dd2-469e-be8b-8f5cf32f0d08 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.629644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501fd76-9dd2-469e-be8b-8f5cf32f0d08 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.629985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501fd76-9dd2-469e-be8b-8f5cf32f0d08 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.631455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 451/902] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:48.637300 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099536e-bfbb-48ea-9edb-2aeefa6f836c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.637610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099536e-bfbb-48ea-9edb-2aeefa6f836c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.637880 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099536e-bfbb-48ea-9edb-2aeefa6f836c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.639431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:48.659921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6099536e-bfbb-48ea-9edb-2aeefa6f836c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:48.661256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6099536e-bfbb-48ea-9edb-2aeefa6f836c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:48.680779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964e91c-e4d8-41dd-a8cd-4be87cd482c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:48.683022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964e91c-e4d8-41dd-a8cd-4be87cd482c4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.683806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964e91c-e4d8-41dd-a8cd-4be87cd482c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.684402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964e91c-e4d8-41dd-a8cd-4be87cd482c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.685537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964e91c-e4d8-41dd-a8cd-4be87cd482c4 None admin] RBAC: Adding query filter params (role.id=437c0e6d42454149ba3b095971b8a9ee, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=f9d0b6662bf947cba853388819a933b8) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:48.686531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964e91c-e4d8-41dd-a8cd-4be87cd482c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:48.689810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964e91c-e4d8-41dd-a8cd-4be87cd482c4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:48.698705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 452/903] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:48 2022] GET /identity/v3/role_assignments?user.id=f9d0b6662bf947cba853388819a933b8&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=437c0e6d42454149ba3b095971b8a9ee => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:48.731058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:48.764342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 452/904] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:05:48.954876 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dba3b-114c-4d52-84e3-2e5dc3ef2f2b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.955408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dba3b-114c-4d52-84e3-2e5dc3ef2f2b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.955408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dba3b-114c-4d52-84e3-2e5dc3ef2f2b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.957092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 453/905] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:48.963665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e47d05-e6be-46e8-bc85-2a79a3b270b3 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:48.964164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e47d05-e6be-46e8-bc85-2a79a3b270b3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:48.964368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e47d05-e6be-46e8-bc85-2a79a3b270b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:48.986080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d7e47d05-e6be-46e8-bc85-2a79a3b270b3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:48.987340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d7e47d05-e6be-46e8-bc85-2a79a3b270b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:49.042548 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dba3b-114c-4d52-84e3-2e5dc3ef2f2b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:49.077337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 453/906] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:49.166970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dba3b-114c-4d52-84e3-2e5dc3ef2f2b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:49.212200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a12ce8-fe0b-4eb0-8369-a414d4a9d25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:49.212920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a12ce8-fe0b-4eb0-8369-a414d4a9d25b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:49.213420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a12ce8-fe0b-4eb0-8369-a414d4a9d25b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:49.213785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a12ce8-fe0b-4eb0-8369-a414d4a9d25b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:49.214530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a12ce8-fe0b-4eb0-8369-a414d4a9d25b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:49.216086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a12ce8-fe0b-4eb0-8369-a414d4a9d25b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e1a12ce8-fe0b-4eb0-8369-a414d4a9d25b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:49.227502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:49.229358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:49.230374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 454/907] 10.222.0.83 () {56 vars in 1132 bytes} [Wed Aug 24 11:05:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:49.238355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e47d05-e6be-46e8-bc85-2a79a3b270b3 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:49.357594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e47d05-e6be-46e8-bc85-2a79a3b270b3 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:49.368169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:49.386326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a12ce8-fe0b-4eb0-8369-a414d4a9d25b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:49.399071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42304480-c0d6-40ce-a906-811747b6c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:49.399891 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42304480-c0d6-40ce-a906-811747b6c178 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:49.400166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42304480-c0d6-40ce-a906-811747b6c178 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:49.400433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42304480-c0d6-40ce-a906-811747b6c178 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:49.401094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42304480-c0d6-40ce-a906-811747b6c178 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:49.401393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42304480-c0d6-40ce-a906-811747b6c178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:49.402973 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42304480-c0d6-40ce-a906-811747b6c178 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:49.410022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 454/908] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:49.418456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42304480-c0d6-40ce-a906-811747b6c178 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:49.464649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:49.563068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a12ce8-fe0b-4eb0-8369-a414d4a9d25b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:49.580923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42304480-c0d6-40ce-a906-811747b6c178 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:49.614371 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3185fa-f409-4272-87e4-6295c8a7a908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:49.614982 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3185fa-f409-4272-87e4-6295c8a7a908 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:49.615332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3185fa-f409-4272-87e4-6295c8a7a908 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:49.615604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3185fa-f409-4272-87e4-6295c8a7a908 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:49.616406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3185fa-f409-4272-87e4-6295c8a7a908 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:49.617947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3185fa-f409-4272-87e4-6295c8a7a908 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:49.628885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c79690-b309-4c7a-aee4-656742f0953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:49.629704 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c79690-b309-4c7a-aee4-656742f0953d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:49.630032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c79690-b309-4c7a-aee4-656742f0953d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:49.630355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c79690-b309-4c7a-aee4-656742f0953d None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:49.631114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c79690-b309-4c7a-aee4-656742f0953d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:49.637512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c79690-b309-4c7a-aee4-656742f0953d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:49.637512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c79690-b309-4c7a-aee4-656742f0953d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:49.638738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 455/909] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:49.645222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c79690-b309-4c7a-aee4-656742f0953d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:49.683978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:49.710046 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 455/910] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:05:49 2022] POST /identity/v3/services => generated 241 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:49.807259 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c79690-b309-4c7a-aee4-656742f0953d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:49.866940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f02a5-1107-4ca4-aac8-14495b412e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:49.867494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f02a5-1107-4ca4-aac8-14495b412e85 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:49.867773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f02a5-1107-4ca4-aac8-14495b412e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:49.868042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f02a5-1107-4ca4-aac8-14495b412e85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:49.868791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f02a5-1107-4ca4-aac8-14495b412e85 None admin] RBAC: Adding query filter params (interface=public, service_id=12ccba205fba4dbcb4e9522af0e69c78, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:49.869097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f02a5-1107-4ca4-aac8-14495b412e85 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:49.870609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f02a5-1107-4ca4-aac8-14495b412e85 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:49.877514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 456/911] 10.222.0.83 () {56 vars in 1251 bytes} [Wed Aug 24 11:05:49 2022] GET /identity/v3/endpoints?service_id=12ccba205fba4dbcb4e9522af0e69c78&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:49.883532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3185fa-f409-4272-87e4-6295c8a7a908 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:50.012871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:50.017427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3185fa-f409-4272-87e4-6295c8a7a908 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:50.053533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bbddf-2602-408c-835d-aa285cc2f2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:50.054169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bbddf-2602-408c-835d-aa285cc2f2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:50.055301 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bbddf-2602-408c-835d-aa285cc2f2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:50.055301 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bbddf-2602-408c-835d-aa285cc2f2a0 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:50.055479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bbddf-2602-408c-835d-aa285cc2f2a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:50.057051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bbddf-2602-408c-835d-aa285cc2f2a0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:50.064682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 456/912] 10.222.0.83 () {56 vars in 1087 bytes} [Wed Aug 24 11:05:49 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:05:50.676484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc1afa-4e84-45cc-b806-73dfa651f48d None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:50.677307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc1afa-4e84-45cc-b806-73dfa651f48d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:50.677724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc1afa-4e84-45cc-b806-73dfa651f48d None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:50.679505 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 457/913] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:50.685496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda667a8-f4bb-4a23-bdc3-b8278ef76f86 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:50.686128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda667a8-f4bb-4a23-bdc3-b8278ef76f86 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:50.687167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda667a8-f4bb-4a23-bdc3-b8278ef76f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:50.706289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fda667a8-f4bb-4a23-bdc3-b8278ef76f86 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:50.707845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fda667a8-f4bb-4a23-bdc3-b8278ef76f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:50.781048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 457/914] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:50.799298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459ee0c-0730-4e3f-a513-303afaf68f1f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:50.799902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459ee0c-0730-4e3f-a513-303afaf68f1f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:50.800380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459ee0c-0730-4e3f-a513-303afaf68f1f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:50.802187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 458/915] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:50.810606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ffb03-3053-4d7a-b7b7-0f530f6b3224 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:50.810894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ffb03-3053-4d7a-b7b7-0f530f6b3224 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:50.811175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ffb03-3053-4d7a-b7b7-0f530f6b3224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:50.830574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-af9ffb03-3053-4d7a-b7b7-0f530f6b3224 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:50.831774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-af9ffb03-3053-4d7a-b7b7-0f530f6b3224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:50.907848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 458/916] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:51.065978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459ee0c-0730-4e3f-a513-303afaf68f1f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:51.169549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:51.177160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459ee0c-0730-4e3f-a513-303afaf68f1f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:51.182779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ffb03-3053-4d7a-b7b7-0f530f6b3224 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:51.215902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:51.216677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a9ffe-de50-484a-bbaa-d90eeeb39807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:51.218228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a9ffe-de50-484a-bbaa-d90eeeb39807 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:51.218528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a9ffe-de50-484a-bbaa-d90eeeb39807 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:51.218803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a9ffe-de50-484a-bbaa-d90eeeb39807 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:51.219418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a9ffe-de50-484a-bbaa-d90eeeb39807 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:51.221735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a9ffe-de50-484a-bbaa-d90eeeb39807 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-fa9a9ffe-de50-484a-bbaa-d90eeeb39807 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:51.234307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:51.248959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:51.249801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 459/917] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:05:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:51.250619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a9ffe-de50-484a-bbaa-d90eeeb39807 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:51.312824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ffb03-3053-4d7a-b7b7-0f530f6b3224 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:51.350023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e3224-c8cf-42a9-9318-50b07f60aaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:51.350803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e3224-c8cf-42a9-9318-50b07f60aaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:51.350912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e3224-c8cf-42a9-9318-50b07f60aaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:51.351191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e3224-c8cf-42a9-9318-50b07f60aaf7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:51.362494 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e3224-c8cf-42a9-9318-50b07f60aaf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:51.364117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a9ffe-de50-484a-bbaa-d90eeeb39807 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b26e3224-c8cf-42a9-9318-50b07f60aaf7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:51.367352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:51.368361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:51.369378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 459/918] 10.222.0.83 () {56 vars in 1120 bytes} [Wed Aug 24 11:05:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:51.376309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e3224-c8cf-42a9-9318-50b07f60aaf7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:51.415928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9522063-b5ee-4a5a-a39d-7e940c8eaa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:51.416545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9522063-b5ee-4a5a-a39d-7e940c8eaa4b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:51.416834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9522063-b5ee-4a5a-a39d-7e940c8eaa4b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:51.417121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9522063-b5ee-4a5a-a39d-7e940c8eaa4b None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:51.417958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9522063-b5ee-4a5a-a39d-7e940c8eaa4b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:51.418239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9522063-b5ee-4a5a-a39d-7e940c8eaa4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:51.420352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9522063-b5ee-4a5a-a39d-7e940c8eaa4b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:51.429983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 460/919] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:05:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:51.434446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9522063-b5ee-4a5a-a39d-7e940c8eaa4b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:51.469650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:51.501400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:51.523394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e3224-c8cf-42a9-9318-50b07f60aaf7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:51.567908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077c09c-1d36-4387-b35a-e7da0832e9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:51.568495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9522063-b5ee-4a5a-a39d-7e940c8eaa4b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:51.569363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077c09c-1d36-4387-b35a-e7da0832e9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:51.570246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077c09c-1d36-4387-b35a-e7da0832e9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:51.570893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077c09c-1d36-4387-b35a-e7da0832e9c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:51.573645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077c09c-1d36-4387-b35a-e7da0832e9c4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:51.574581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077c09c-1d36-4387-b35a-e7da0832e9c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:51.578241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077c09c-1d36-4387-b35a-e7da0832e9c4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:51.600435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 460/920] 10.222.0.83 () {56 vars in 1115 bytes} [Wed Aug 24 11:05:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:51.602026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077c09c-1d36-4387-b35a-e7da0832e9c4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:51.622007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd066516-b7aa-48ba-b6f0-5ae58feeed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:51.622560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd066516-b7aa-48ba-b6f0-5ae58feeed1c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:51.622840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd066516-b7aa-48ba-b6f0-5ae58feeed1c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:51.623114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd066516-b7aa-48ba-b6f0-5ae58feeed1c None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:51.623763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd066516-b7aa-48ba-b6f0-5ae58feeed1c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:51.624070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd066516-b7aa-48ba-b6f0-5ae58feeed1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:51.626491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd066516-b7aa-48ba-b6f0-5ae58feeed1c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:51.637989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 461/921] 10.222.0.83 () {56 vars in 1114 bytes} [Wed Aug 24 11:05:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:51.640800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd066516-b7aa-48ba-b6f0-5ae58feeed1c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:51.747961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077c09c-1d36-4387-b35a-e7da0832e9c4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:51.770462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:51.785823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd066516-b7aa-48ba-b6f0-5ae58feeed1c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:51.793008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8b1b0-36bc-442d-bdbc-edbba7da5a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:51.793975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8b1b0-36bc-442d-bdbc-edbba7da5a64 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:51.794484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8b1b0-36bc-442d-bdbc-edbba7da5a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:51.794958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8b1b0-36bc-442d-bdbc-edbba7da5a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:51.810904 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8b1b0-36bc-442d-bdbc-edbba7da5a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:51.813502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8b1b0-36bc-442d-bdbc-edbba7da5a64 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:51.822585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:51.828114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a65b1-1153-489b-9daa-1bd97654f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:51.828797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a65b1-1153-489b-9daa-1bd97654f174 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:51.829017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a65b1-1153-489b-9daa-1bd97654f174 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:51.829290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a65b1-1153-489b-9daa-1bd97654f174 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:51.830010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a65b1-1153-489b-9daa-1bd97654f174 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:51.831394 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a65b1-1153-489b-9daa-1bd97654f174 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a7f8b1b0-36bc-442d-bdbc-edbba7da5a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:51.834188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:51.841173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:51.842074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:51.842074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:51.842074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:51.842900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 461/922] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:51.844392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8b1b0-36bc-442d-bdbc-edbba7da5a64 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:51.926290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 462/923] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:05:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:05:51.927969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daa5e0-2f7a-491f-a75e-b02db436659c None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:51.928322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daa5e0-2f7a-491f-a75e-b02db436659c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:51.928591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daa5e0-2f7a-491f-a75e-b02db436659c None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:51.934917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 463/924] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:51.936900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55b1b-b288-44bb-8395-1c026439943d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:51.937168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55b1b-b288-44bb-8395-1c026439943d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:51.937441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55b1b-b288-44bb-8395-1c026439943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:51.958655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5ed55b1b-b288-44bb-8395-1c026439943d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:51.959904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5ed55b1b-b288-44bb-8395-1c026439943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:51.981759 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8b1b0-36bc-442d-bdbc-edbba7da5a64 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:52.032271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a124b61-8bf2-4ae5-87ba-29401751bda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:52.033222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a124b61-8bf2-4ae5-87ba-29401751bda4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.033715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a124b61-8bf2-4ae5-87ba-29401751bda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.034198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a124b61-8bf2-4ae5-87ba-29401751bda4 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.035239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a124b61-8bf2-4ae5-87ba-29401751bda4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:52.035663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a124b61-8bf2-4ae5-87ba-29401751bda4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:52.037243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a124b61-8bf2-4ae5-87ba-29401751bda4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:52.052897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 462/925] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:52.059086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a124b61-8bf2-4ae5-87ba-29401751bda4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:52.059697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 464/926] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:52.107644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48f572-209d-4f1e-a82e-fcadc85d79a2 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.107856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48f572-209d-4f1e-a82e-fcadc85d79a2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.110233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48f572-209d-4f1e-a82e-fcadc85d79a2 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.111828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 465/927] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:05:52.117905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7354c57-1ceb-4d98-87ed-8df5777e1489 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.118213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7354c57-1ceb-4d98-87ed-8df5777e1489 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.121341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7354c57-1ceb-4d98-87ed-8df5777e1489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.131744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:52.139657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b7354c57-1ceb-4d98-87ed-8df5777e1489 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:52.140886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b7354c57-1ceb-4d98-87ed-8df5777e1489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:52.191804 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a124b61-8bf2-4ae5-87ba-29401751bda4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:52.242352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e497a4-4fdc-4789-a8ba-25c747ded6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:52.243649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e497a4-4fdc-4789-a8ba-25c747ded6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.244142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e497a4-4fdc-4789-a8ba-25c747ded6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.244848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e497a4-4fdc-4789-a8ba-25c747ded6b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.255647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 466/928] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:52.256083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e497a4-4fdc-4789-a8ba-25c747ded6b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:52.257546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e497a4-4fdc-4789-a8ba-25c747ded6b3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-07e497a4-4fdc-4789-a8ba-25c747ded6b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:52.273525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:52.274579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:52.275539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 463/929] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:05:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:52.280590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7354c57-1ceb-4d98-87ed-8df5777e1489 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:52.286472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:52.381147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e497a4-4fdc-4789-a8ba-25c747ded6b3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:52.397576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7354c57-1ceb-4d98-87ed-8df5777e1489 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:52.440173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f49700-08cc-4b13-aec7-f183780445cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:52.440792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f49700-08cc-4b13-aec7-f183780445cf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.441068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f49700-08cc-4b13-aec7-f183780445cf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.441337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f49700-08cc-4b13-aec7-f183780445cf None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.442547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f49700-08cc-4b13-aec7-f183780445cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:52.442860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f49700-08cc-4b13-aec7-f183780445cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:52.444858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f49700-08cc-4b13-aec7-f183780445cf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:52.470240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 467/930] 10.222.0.83 () {56 vars in 1149 bytes} [Wed Aug 24 11:05:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:52.478907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f49700-08cc-4b13-aec7-f183780445cf None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:52.509530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e497a4-4fdc-4789-a8ba-25c747ded6b3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:52.527349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:52.551657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ff9a6-1254-47bb-8b79-bd771faa7fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:52.552292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ff9a6-1254-47bb-8b79-bd771faa7fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.552574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ff9a6-1254-47bb-8b79-bd771faa7fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.552855 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ff9a6-1254-47bb-8b79-bd771faa7fa1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.553448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ff9a6-1254-47bb-8b79-bd771faa7fa1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:52.554848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ff9a6-1254-47bb-8b79-bd771faa7fa1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:52.557207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b08ff9a6-1254-47bb-8b79-bd771faa7fa1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:52.564854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:52.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:52.566796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 464/931] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:52.567594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ff9a6-1254-47bb-8b79-bd771faa7fa1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:52.604046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f49700-08cc-4b13-aec7-f183780445cf None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:52.645454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4fe45-4ea0-4add-96a0-76e96c7c1fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:52.646037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4fe45-4ea0-4add-96a0-76e96c7c1fbc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.646312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4fe45-4ea0-4add-96a0-76e96c7c1fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.646583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4fe45-4ea0-4add-96a0-76e96c7c1fbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.659459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4fe45-4ea0-4add-96a0-76e96c7c1fbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:52.661643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4fe45-4ea0-4add-96a0-76e96c7c1fbc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:52.680712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ff9a6-1254-47bb-8b79-bd771faa7fa1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d3a4fe45-4ea0-4add-96a0-76e96c7c1fbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:52.681996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:52.682946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:52.683885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:52.683885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:52.683885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:52.684617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 468/932] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:52.689744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4fe45-4ea0-4add-96a0-76e96c7c1fbc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:52.721241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92b040-39d7-4a44-8a77-796ab3d221bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:52.722265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92b040-39d7-4a44-8a77-796ab3d221bb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.722579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92b040-39d7-4a44-8a77-796ab3d221bb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.722848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92b040-39d7-4a44-8a77-796ab3d221bb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.723433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92b040-39d7-4a44-8a77-796ab3d221bb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:52.725042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92b040-39d7-4a44-8a77-796ab3d221bb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0f92b040-39d7-4a44-8a77-796ab3d221bb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:52.733905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:52.734873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:52.735768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 465/933] 10.222.0.83 () {56 vars in 1132 bytes} [Wed Aug 24 11:05:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:52.736597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92b040-39d7-4a44-8a77-796ab3d221bb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:52.784555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:52.807822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4fe45-4ea0-4add-96a0-76e96c7c1fbc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:52.822933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:52.843798 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fbae5-784b-40d7-ae22-03c636e28c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:52.844447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fbae5-784b-40d7-ae22-03c636e28c83 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.844766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fbae5-784b-40d7-ae22-03c636e28c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.845018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fbae5-784b-40d7-ae22-03c636e28c83 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.845662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fbae5-784b-40d7-ae22-03c636e28c83 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:52.846020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fbae5-784b-40d7-ae22-03c636e28c83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:52.847415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fbae5-784b-40d7-ae22-03c636e28c83 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:52.849600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92b040-39d7-4a44-8a77-796ab3d221bb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:52.852419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 469/934] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:05:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:52.853250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fbae5-784b-40d7-ae22-03c636e28c83 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:52.891107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcca45-8a59-4eed-82e1-7a3d4084abb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:52.891972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcca45-8a59-4eed-82e1-7a3d4084abb0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:52.892246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcca45-8a59-4eed-82e1-7a3d4084abb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:52.892527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcca45-8a59-4eed-82e1-7a3d4084abb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:52.893164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcca45-8a59-4eed-82e1-7a3d4084abb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:52.893461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcca45-8a59-4eed-82e1-7a3d4084abb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:52.894913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcca45-8a59-4eed-82e1-7a3d4084abb0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:52.913456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 466/935] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:52.913456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcca45-8a59-4eed-82e1-7a3d4084abb0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:52.986480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fbae5-784b-40d7-ae22-03c636e28c83 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:53.021003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcca45-8a59-4eed-82e1-7a3d4084abb0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:53.022818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:53.036923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca7b8d-119d-434d-b171-dac31d5220cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:53.037961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca7b8d-119d-434d-b171-dac31d5220cb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:53.038340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca7b8d-119d-434d-b171-dac31d5220cb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:53.038674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca7b8d-119d-434d-b171-dac31d5220cb None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:53.039460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca7b8d-119d-434d-b171-dac31d5220cb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:53.039833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca7b8d-119d-434d-b171-dac31d5220cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:53.041413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca7b8d-119d-434d-b171-dac31d5220cb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:53.051288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 470/936] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:53.051288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca7b8d-119d-434d-b171-dac31d5220cb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:53.065507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d137d5-2aa1-448a-8372-c12e4dfc3c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:53.066154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d137d5-2aa1-448a-8372-c12e4dfc3c5f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:53.066436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d137d5-2aa1-448a-8372-c12e4dfc3c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:53.066709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d137d5-2aa1-448a-8372-c12e4dfc3c5f None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:53.067349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d137d5-2aa1-448a-8372-c12e4dfc3c5f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:53.067652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d137d5-2aa1-448a-8372-c12e4dfc3c5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:53.069122 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d137d5-2aa1-448a-8372-c12e4dfc3c5f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:53.075370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 467/937] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:05:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:53.075370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d137d5-2aa1-448a-8372-c12e4dfc3c5f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:53.107041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:53.175243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca7b8d-119d-434d-b171-dac31d5220cb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:53.210235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d137d5-2aa1-448a-8372-c12e4dfc3c5f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:53.214613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d5a26-d85f-4da3-b59f-91db2a2b4605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:53.215251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d5a26-d85f-4da3-b59f-91db2a2b4605 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:53.215541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d5a26-d85f-4da3-b59f-91db2a2b4605 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:53.215801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d5a26-d85f-4da3-b59f-91db2a2b4605 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:53.223800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d5a26-d85f-4da3-b59f-91db2a2b4605 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:53.225274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d5a26-d85f-4da3-b59f-91db2a2b4605 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3b1d5a26-d85f-4da3-b59f-91db2a2b4605 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:53.236972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:53.238127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:53.239025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:53.239025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:53.239025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:53.239025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:53.239025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:53.239025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:53.239025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:53.239025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:53.239025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 471/938] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:53.239522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d5a26-d85f-4da3-b59f-91db2a2b4605 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:53.255424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ffd59-8ee0-4613-bbf2-3f7c374ee233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:53.256346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ffd59-8ee0-4613-bbf2-3f7c374ee233 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:53.256346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ffd59-8ee0-4613-bbf2-3f7c374ee233 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:53.256605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ffd59-8ee0-4613-bbf2-3f7c374ee233 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:53.257234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ffd59-8ee0-4613-bbf2-3f7c374ee233 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:53.257544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ffd59-8ee0-4613-bbf2-3f7c374ee233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:53.258938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ffd59-8ee0-4613-bbf2-3f7c374ee233 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:53.264147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 468/939] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:53.267037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ffd59-8ee0-4613-bbf2-3f7c374ee233 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:53.303636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:53.356583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d5a26-d85f-4da3-b59f-91db2a2b4605 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:53.358210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:53.387027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ffd59-8ee0-4613-bbf2-3f7c374ee233 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:53.393403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205350d-b484-4af0-ae26-9060856e5a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:53.394060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205350d-b484-4af0-ae26-9060856e5a1d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:53.394343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205350d-b484-4af0-ae26-9060856e5a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:53.394701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205350d-b484-4af0-ae26-9060856e5a1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:53.395340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205350d-b484-4af0-ae26-9060856e5a1d None admin] RBAC: Adding query filter params (interface=public, service_id=3b68442c093e4ce2bfcd59ed3d8826f4, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:53.395710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205350d-b484-4af0-ae26-9060856e5a1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:53.397228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205350d-b484-4af0-ae26-9060856e5a1d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:53.403721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 472/940] 10.222.0.83 () {56 vars in 1251 bytes} [Wed Aug 24 11:05:53 2022] GET /identity/v3/endpoints?service_id=3b68442c093e4ce2bfcd59ed3d8826f4&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:53.406803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205350d-b484-4af0-ae26-9060856e5a1d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:53.427215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5750c66-43af-4c98-896b-d7eb4d730e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:53.427754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5750c66-43af-4c98-896b-d7eb4d730e91 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:53.428034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5750c66-43af-4c98-896b-d7eb4d730e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:53.428308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5750c66-43af-4c98-896b-d7eb4d730e91 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:53.429007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5750c66-43af-4c98-896b-d7eb4d730e91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:53.429311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5750c66-43af-4c98-896b-d7eb4d730e91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:53.430800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5750c66-43af-4c98-896b-d7eb4d730e91 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:53.444122 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 469/941] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:53.444975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5750c66-43af-4c98-896b-d7eb4d730e91 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:53.526167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205350d-b484-4af0-ae26-9060856e5a1d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:53.545755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:53.557133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5750c66-43af-4c98-896b-d7eb4d730e91 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:53.566872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad7617-4277-4c48-8f2d-c89e73a80d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:53.567526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad7617-4277-4c48-8f2d-c89e73a80d24 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:53.567811 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad7617-4277-4c48-8f2d-c89e73a80d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:53.568086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad7617-4277-4c48-8f2d-c89e73a80d24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:53.568756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad7617-4277-4c48-8f2d-c89e73a80d24 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:53.570178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad7617-4277-4c48-8f2d-c89e73a80d24 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:53.594421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d8628-703b-42e0-8be0-35884e9a6556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:53.595022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d8628-703b-42e0-8be0-35884e9a6556 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:53.595299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d8628-703b-42e0-8be0-35884e9a6556 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:53.595570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d8628-703b-42e0-8be0-35884e9a6556 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:53.598346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d8628-703b-42e0-8be0-35884e9a6556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:53.598346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d8628-703b-42e0-8be0-35884e9a6556 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:53.603928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 470/942] 10.222.0.83 () {56 vars in 1087 bytes} [Wed Aug 24 11:05:53 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:05:53.604718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d8628-703b-42e0-8be0-35884e9a6556 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:53.706506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 473/943] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:05:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:53.743747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d8628-703b-42e0-8be0-35884e9a6556 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:53.799941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18081c18-313b-44fd-859e-d935e702bdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:53.800997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18081c18-313b-44fd-859e-d935e702bdcd None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:53.801491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18081c18-313b-44fd-859e-d935e702bdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:53.802004 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18081c18-313b-44fd-859e-d935e702bdcd None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/f9d0b6662bf947cba853388819a933b8/roles/437c0e6d42454149ba3b095971b8a9ee` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:53.804713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18081c18-313b-44fd-859e-d935e702bdcd None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=f9d0b6662bf947cba853388819a933b8, role_id=437c0e6d42454149ba3b095971b8a9ee)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:53.806978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18081c18-313b-44fd-859e-d935e702bdcd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:53.875442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 471/944] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:05:53 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/f9d0b6662bf947cba853388819a933b8/roles/437c0e6d42454149ba3b095971b8a9ee => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:05:54.027179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4406f-39e8-44c2-9a04-4cd0c1c272e5 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:54.027667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4406f-39e8-44c2-9a04-4cd0c1c272e5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:54.027667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4406f-39e8-44c2-9a04-4cd0c1c272e5 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:54.029126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 474/945] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:54.035416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898efae-baaf-41b3-b583-3ee5ff90d5f9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:54.035999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898efae-baaf-41b3-b583-3ee5ff90d5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:54.036478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898efae-baaf-41b3-b583-3ee5ff90d5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:54.057934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d898efae-baaf-41b3-b583-3ee5ff90d5f9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:54.059152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d898efae-baaf-41b3-b583-3ee5ff90d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:54.064348 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:54.136950 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 472/946] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:54.404260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4406f-39e8-44c2-9a04-4cd0c1c272e5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:54.432376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:54.515335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4406f-39e8-44c2-9a04-4cd0c1c272e5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:54.555860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:54.557153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:54.557636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:54.558135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:54.559083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:54.560740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:54.564157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING keystone.server.flask.common [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 11:05:54.621581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.sql.core [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 11:05:54.621581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 11:05:54.621581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [parameters: {'id': 'c9d671e5b546456bb757b27e67677833', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 11:05:54.621581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81864) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c9d671e5b546456bb757b27e67677833.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c9d671e5b546456bb757b27e67677833. Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:05:54.639549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application result.read() Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 11:05:54.640424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application yield resource Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise self.value Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application session.commit() Aug 24 11:05:54.641376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.session.flush() Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application compat.raise_( Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 11:05:54.642286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 11:05:54.646097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application result.read() Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:05:54.647200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [parameters: {'id': 'c9d671e5b546456bb757b27e67677833', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:54.648055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:54.648847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:54.655068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:54.655068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 11:05:54.655068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 11:05:54.655068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:54.655068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:54.655068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 11:05:54.655068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 11:05:54.655068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c9d671e5b546456bb757b27e67677833. Aug 24 11:05:54.655068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:54.656609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 475/947] 10.222.0.83 () {60 vars in 1140 bytes} [Wed Aug 24 11:05:54 2022] POST /identity/v3/projects => generated 262 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:54.663731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898efae-baaf-41b3-b583-3ee5ff90d5f9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:54.777332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898efae-baaf-41b3-b583-3ee5ff90d5f9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:54.816804 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdc604-7dbd-4a0c-95e3-cb684350c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:54.817834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdc604-7dbd-4a0c-95e3-cb684350c963 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:54.818355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdc604-7dbd-4a0c-95e3-cb684350c963 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:54.818910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdc604-7dbd-4a0c-95e3-cb684350c963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:54.828300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdc604-7dbd-4a0c-95e3-cb684350c963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:54.830290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdc604-7dbd-4a0c-95e3-cb684350c963 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:54.833540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a9bdc604-7dbd-4a0c-95e3-cb684350c963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:54.843881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:54.850564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:54.851495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:54.851495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:54.851495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:54.851495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:54.851495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:54.851495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:54.851495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:54.851495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:54.851495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 473/948] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:54.854984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:54.947147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:54.968685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330911bd-142e-4d31-ab75-02dc13bcc0c8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:55.005786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e28df1-d09c-498d-aa0d-354a20171daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:55.006443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e28df1-d09c-498d-aa0d-354a20171daa None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:55.006794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e28df1-d09c-498d-aa0d-354a20171daa None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:55.007076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e28df1-d09c-498d-aa0d-354a20171daa None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:55.007770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e28df1-d09c-498d-aa0d-354a20171daa None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:55.008077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e28df1-d09c-498d-aa0d-354a20171daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:55.009802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e28df1-d09c-498d-aa0d-354a20171daa None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:55.039965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 476/949] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:05:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:55.705711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43595177-4190-4413-9588-bda7cc3b6b76 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:55.706277 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43595177-4190-4413-9588-bda7cc3b6b76 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:55.706277 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43595177-4190-4413-9588-bda7cc3b6b76 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:55.707999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 474/950] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:05:55.714113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9178fda-58e9-4e6c-9964-cef56d180f2c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:55.714451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9178fda-58e9-4e6c-9964-cef56d180f2c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:55.714679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9178fda-58e9-4e6c-9964-cef56d180f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:55.737464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f9178fda-58e9-4e6c-9964-cef56d180f2c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:55.739210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f9178fda-58e9-4e6c-9964-cef56d180f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:55.752193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63771d9-4524-4ea8-b143-6aab72c5dea9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:55.752824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63771d9-4524-4ea8-b143-6aab72c5dea9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:55.753471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63771d9-4524-4ea8-b143-6aab72c5dea9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:55.755554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 475/951] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:05:55.761821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281620f8-3c74-4497-a5ff-42c924423b01 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:55.762522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281620f8-3c74-4497-a5ff-42c924423b01 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:55.763027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281620f8-3c74-4497-a5ff-42c924423b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:55.784785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-281620f8-3c74-4497-a5ff-42c924423b01 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:55.786516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-281620f8-3c74-4497-a5ff-42c924423b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:55.841380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 477/952] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:55.876710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 476/953] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:05:55.907861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609971e-8dcf-4346-b202-878574a1b742 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:55.908182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609971e-8dcf-4346-b202-878574a1b742 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:55.908426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609971e-8dcf-4346-b202-878574a1b742 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:55.910083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 478/954] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:55.915475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18641c39-8a65-47d4-9327-c57de39a0080 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:55.915761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18641c39-8a65-47d4-9327-c57de39a0080 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:55.916092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18641c39-8a65-47d4-9327-c57de39a0080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:55.935950 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-18641c39-8a65-47d4-9327-c57de39a0080 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:55.937338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-18641c39-8a65-47d4-9327-c57de39a0080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:56.010871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 477/955] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:56.117024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609971e-8dcf-4346-b202-878574a1b742 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.155730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18641c39-8a65-47d4-9327-c57de39a0080 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.229052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609971e-8dcf-4346-b202-878574a1b742 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:56.236513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:56.239904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:56.268829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b62f09-23e5-4fd5-84d6-5fcf34a612ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:56.269554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b62f09-23e5-4fd5-84d6-5fcf34a612ba None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:56.269987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b62f09-23e5-4fd5-84d6-5fcf34a612ba None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:56.270377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b62f09-23e5-4fd5-84d6-5fcf34a612ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:56.271261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b62f09-23e5-4fd5-84d6-5fcf34a612ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:56.272753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b62f09-23e5-4fd5-84d6-5fcf34a612ba None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-70b62f09-23e5-4fd5-84d6-5fcf34a612ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:56.287312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:56.293898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:56.298802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 479/956] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:56.299712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b62f09-23e5-4fd5-84d6-5fcf34a612ba None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.299783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18641c39-8a65-47d4-9327-c57de39a0080 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:56.342981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc2e7f-d1f7-42b4-8507-8ff9618cc7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:56.343676 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc2e7f-d1f7-42b4-8507-8ff9618cc7fe None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:56.344089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc2e7f-d1f7-42b4-8507-8ff9618cc7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:56.344512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc2e7f-d1f7-42b4-8507-8ff9618cc7fe None admin] PATH_INFO: `/v3/services/compute` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:56.345137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc2e7f-d1f7-42b4-8507-8ff9618cc7fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:56.347015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc2e7f-d1f7-42b4-8507-8ff9618cc7fe None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-74bc2e7f-d1f7-42b4-8507-8ff9618cc7fe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:56.356623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:56.357666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:56.358586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 478/957] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:56.359468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc2e7f-d1f7-42b4-8507-8ff9618cc7fe None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.412083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b62f09-23e5-4fd5-84d6-5fcf34a612ba None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:56.446067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:56.450953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce6f32-d2a6-4938-b3d0-0a4f2578eecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:56.451567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce6f32-d2a6-4938-b3d0-0a4f2578eecb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:56.451961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce6f32-d2a6-4938-b3d0-0a4f2578eecb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:56.452179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce6f32-d2a6-4938-b3d0-0a4f2578eecb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:56.466576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce6f32-d2a6-4938-b3d0-0a4f2578eecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-8cce6f32-d2a6-4938-b3d0-0a4f2578eecb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:05:56.471645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:56.472578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:56.473416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 480/958] 10.222.0.83 () {56 vars in 1120 bytes} [Wed Aug 24 11:05:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:56.474838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce6f32-d2a6-4938-b3d0-0a4f2578eecb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.497217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc2e7f-d1f7-42b4-8507-8ff9618cc7fe None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:56.526239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:56.538757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8526a-bf5a-41e5-a755-aa6bd899bb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:56.539473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8526a-bf5a-41e5-a755-aa6bd899bb87 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:56.539850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8526a-bf5a-41e5-a755-aa6bd899bb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:56.540141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8526a-bf5a-41e5-a755-aa6bd899bb87 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:56.540763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8526a-bf5a-41e5-a755-aa6bd899bb87 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:56.541065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8526a-bf5a-41e5-a755-aa6bd899bb87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:56.542826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8526a-bf5a-41e5-a755-aa6bd899bb87 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:56.552733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 479/959] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:05:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:56.552733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8526a-bf5a-41e5-a755-aa6bd899bb87 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.592785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce6f32-d2a6-4938-b3d0-0a4f2578eecb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:56.629846 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa62bab-9bf4-42db-8816-2cc9731b4531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:56.630514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa62bab-9bf4-42db-8816-2cc9731b4531 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:56.630846 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa62bab-9bf4-42db-8816-2cc9731b4531 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:56.631154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa62bab-9bf4-42db-8816-2cc9731b4531 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:56.631792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa62bab-9bf4-42db-8816-2cc9731b4531 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:56.632251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa62bab-9bf4-42db-8816-2cc9731b4531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:56.633718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa62bab-9bf4-42db-8816-2cc9731b4531 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:56.643961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 481/960] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:05:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:56.643961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa62bab-9bf4-42db-8816-2cc9731b4531 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.674499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8526a-bf5a-41e5-a755-aa6bd899bb87 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:56.704566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:56.713787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cb396-6157-49c4-a633-4b2fa9469f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:56.717274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cb396-6157-49c4-a633-4b2fa9469f87 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:56.717274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cb396-6157-49c4-a633-4b2fa9469f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:56.717274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cb396-6157-49c4-a633-4b2fa9469f87 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:56.717274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cb396-6157-49c4-a633-4b2fa9469f87 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:56.717274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cb396-6157-49c4-a633-4b2fa9469f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:56.717274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cb396-6157-49c4-a633-4b2fa9469f87 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:56.729914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 480/961] 10.222.0.83 () {56 vars in 1115 bytes} [Wed Aug 24 11:05:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:56.729914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cb396-6157-49c4-a633-4b2fa9469f87 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.760593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa62bab-9bf4-42db-8816-2cc9731b4531 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:56.784712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:56.798168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9809ce-9c6b-44c4-b07a-a6068bb74c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:56.798776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9809ce-9c6b-44c4-b07a-a6068bb74c9a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:56.799053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9809ce-9c6b-44c4-b07a-a6068bb74c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:56.799325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9809ce-9c6b-44c4-b07a-a6068bb74c9a None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:56.802955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9809ce-9c6b-44c4-b07a-a6068bb74c9a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:56.802955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9809ce-9c6b-44c4-b07a-a6068bb74c9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:56.802955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9809ce-9c6b-44c4-b07a-a6068bb74c9a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:56.807457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 482/962] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:05:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:56.808330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9809ce-9c6b-44c4-b07a-a6068bb74c9a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.847722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cb396-6157-49c4-a633-4b2fa9469f87 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:56.884846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbe6a7-607d-4207-8d2d-3dc05578d863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:56.885521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbe6a7-607d-4207-8d2d-3dc05578d863 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:56.885819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbe6a7-607d-4207-8d2d-3dc05578d863 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:56.886121 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbe6a7-607d-4207-8d2d-3dc05578d863 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:56.886753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbe6a7-607d-4207-8d2d-3dc05578d863 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:56.887052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbe6a7-607d-4207-8d2d-3dc05578d863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:56.888477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbe6a7-607d-4207-8d2d-3dc05578d863 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:56.893458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 481/963] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:05:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:56.903175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbe6a7-607d-4207-8d2d-3dc05578d863 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:56.931892 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9809ce-9c6b-44c4-b07a-a6068bb74c9a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:56.965485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:56.989577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3210fec-c4cd-456f-9199-451c475cc018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:56.990582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3210fec-c4cd-456f-9199-451c475cc018 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:56.991360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3210fec-c4cd-456f-9199-451c475cc018 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:56.991926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3210fec-c4cd-456f-9199-451c475cc018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:57.009043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3210fec-c4cd-456f-9199-451c475cc018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:57.011323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3210fec-c4cd-456f-9199-451c475cc018 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f3210fec-c4cd-456f-9199-451c475cc018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:57.033508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:57.034576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:57.035520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:57.035520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:57.035520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:57.035719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbe6a7-607d-4207-8d2d-3dc05578d863 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:57.039292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 483/964] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:57.044983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3210fec-c4cd-456f-9199-451c475cc018 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:57.058361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:57.085808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f650b0b-48e6-47e3-9992-9527b8b111a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:57.086476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f650b0b-48e6-47e3-9992-9527b8b111a5 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:57.086760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f650b0b-48e6-47e3-9992-9527b8b111a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:57.087112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f650b0b-48e6-47e3-9992-9527b8b111a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:57.087830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f650b0b-48e6-47e3-9992-9527b8b111a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:57.089562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f650b0b-48e6-47e3-9992-9527b8b111a5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:57.172775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3210fec-c4cd-456f-9199-451c475cc018 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:57.216495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516681c-abb6-4119-9cbc-fc704cead2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:57.217120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516681c-abb6-4119-9cbc-fc704cead2ed None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:57.217399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516681c-abb6-4119-9cbc-fc704cead2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:57.218278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516681c-abb6-4119-9cbc-fc704cead2ed None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:57.218947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516681c-abb6-4119-9cbc-fc704cead2ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:57.219265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516681c-abb6-4119-9cbc-fc704cead2ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:57.220769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516681c-abb6-4119-9cbc-fc704cead2ed None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:57.226387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 482/965] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:05:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:05:57.227870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277a4fc-7b8a-4881-8964-bdd7f7586cfa None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:57.228156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277a4fc-7b8a-4881-8964-bdd7f7586cfa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:57.228430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277a4fc-7b8a-4881-8964-bdd7f7586cfa None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:57.229943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 483/966] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:57.240417 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 484/967] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:05:57.242874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90c0a1-6d0e-4060-8391-3c2cbefb3f67 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:57.243172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90c0a1-6d0e-4060-8391-3c2cbefb3f67 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:57.243551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90c0a1-6d0e-4060-8391-3c2cbefb3f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:57.247051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516681c-abb6-4119-9cbc-fc704cead2ed None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:57.268344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cb90c0a1-6d0e-4060-8391-3c2cbefb3f67 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:57.269848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cb90c0a1-6d0e-4060-8391-3c2cbefb3f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:57.291034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:57.393914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 484/968] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:05:57.395458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516681c-abb6-4119-9cbc-fc704cead2ed None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:57.447476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef498d8d-5b33-4ce8-a123-351e41d99c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:57.448186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef498d8d-5b33-4ce8-a123-351e41d99c59 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:57.448467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef498d8d-5b33-4ce8-a123-351e41d99c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:57.448739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef498d8d-5b33-4ce8-a123-351e41d99c59 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:57.460354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef498d8d-5b33-4ce8-a123-351e41d99c59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:57.461834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef498d8d-5b33-4ce8-a123-351e41d99c59 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ef498d8d-5b33-4ce8-a123-351e41d99c59 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:57.475257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:57.476310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:57.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 485/969] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:05:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:57.484320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90c0a1-6d0e-4060-8391-3c2cbefb3f67 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:57.585236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:57.591728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90c0a1-6d0e-4060-8391-3c2cbefb3f67 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:57.630090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21985d-5e44-4c1a-9b3c-99ba167361c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:57.631394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21985d-5e44-4c1a-9b3c-99ba167361c1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:57.631935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21985d-5e44-4c1a-9b3c-99ba167361c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:57.632205 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21985d-5e44-4c1a-9b3c-99ba167361c1 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:57.632945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21985d-5e44-4c1a-9b3c-99ba167361c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:57.633247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21985d-5e44-4c1a-9b3c-99ba167361c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:57.634696 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21985d-5e44-4c1a-9b3c-99ba167361c1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:57.653525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 485/970] 10.222.0.83 () {56 vars in 1149 bytes} [Wed Aug 24 11:05:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:57.660688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef498d8d-5b33-4ce8-a123-351e41d99c59 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:57.712283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21985d-5e44-4c1a-9b3c-99ba167361c1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:57.774290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:57.784462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef498d8d-5b33-4ce8-a123-351e41d99c59 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:57.826000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21985d-5e44-4c1a-9b3c-99ba167361c1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:57.830239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4c1ed-2ba5-4927-ac3a-55cb58ff3d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:57.835025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4c1ed-2ba5-4927-ac3a-55cb58ff3d87 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:57.835363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4c1ed-2ba5-4927-ac3a-55cb58ff3d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:57.835574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4c1ed-2ba5-4927-ac3a-55cb58ff3d87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:57.874470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:57.875338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:57.875621 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:57.875888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:57.876541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:57.878060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:57.880757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WARNING keystone.server.flask.common [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 11:05:57.919137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4c1ed-2ba5-4927-ac3a-55cb58ff3d87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:57.920617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4c1ed-2ba5-4927-ac3a-55cb58ff3d87 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:57.935297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.sql.core [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 11:05:57.935297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 11:05:57.935297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [parameters: {'user_id': 'b50ddcfdbac5426e86c08f1d0f385de0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 11:05:57.935297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81865) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result.read() Aug 24 11:05:57.945690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:05:57.946998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application yield resource Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise self.value Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application session.commit() Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 11:05:57.948193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.session.flush() Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application compat.raise_( Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 11:05:57.949311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:05:57.950662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result.read() Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:05:57.951871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b50ddcfdbac5426e86c08f1d0f385de0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:57.953590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 11:05:57.955171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 11:05:57.958422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 11:05:57.958422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 11:05:57.958422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 11:05:57.958422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 11:05:57.958422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 11:05:57.958422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:57.960024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 486/971] 10.222.0.83 () {60 vars in 1131 bytes} [Wed Aug 24 11:05:57 2022] POST /identity/v3/users => generated 160 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:57.967286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:57.996845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:58.046480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:58.053749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 486/972] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:05:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:05:58.058906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4c1ed-2ba5-4927-ac3a-55cb58ff3d87 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:58.110590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6fe14-bfd6-45e2-80be-45cb608e6c45 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:58.150303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17631e4f-e8db-4c84-b786-167bdf0f5d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:58.151161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17631e4f-e8db-4c84-b786-167bdf0f5d7a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:58.151511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17631e4f-e8db-4c84-b786-167bdf0f5d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:58.151969 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17631e4f-e8db-4c84-b786-167bdf0f5d7a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:58.160665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17631e4f-e8db-4c84-b786-167bdf0f5d7a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:58.162559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17631e4f-e8db-4c84-b786-167bdf0f5d7a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-17631e4f-e8db-4c84-b786-167bdf0f5d7a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:05:58.176273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:58.177299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:05:58.178320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:58.182399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 487/973] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:05:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:58.184462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17631e4f-e8db-4c84-b786-167bdf0f5d7a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:58.194176 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4c1ed-2ba5-4927-ac3a-55cb58ff3d87 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:58.247903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df981d6-f800-424b-84bf-7f85aa40d538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:58.249424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df981d6-f800-424b-84bf-7f85aa40d538 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:58.250230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df981d6-f800-424b-84bf-7f85aa40d538 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:58.250728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df981d6-f800-424b-84bf-7f85aa40d538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:58.266657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df981d6-f800-424b-84bf-7f85aa40d538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:58.269552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df981d6-f800-424b-84bf-7f85aa40d538 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:58.277651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-0df981d6-f800-424b-84bf-7f85aa40d538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:05:58.289954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:58.291566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:05:58.292464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:05:58.292464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:05:58.292464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:58.299340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 487/974] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:58.301131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df981d6-f800-424b-84bf-7f85aa40d538 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:58.315886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17631e4f-e8db-4c84-b786-167bdf0f5d7a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:58.356583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:58.370528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c538cb7-2126-427a-85b3-aa5e8678192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:58.370528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c538cb7-2126-427a-85b3-aa5e8678192f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:58.370528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c538cb7-2126-427a-85b3-aa5e8678192f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:58.370528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c538cb7-2126-427a-85b3-aa5e8678192f None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:58.371011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c538cb7-2126-427a-85b3-aa5e8678192f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:58.371011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c538cb7-2126-427a-85b3-aa5e8678192f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:58.371011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c538cb7-2126-427a-85b3-aa5e8678192f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:58.421904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df981d6-f800-424b-84bf-7f85aa40d538 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:58.422563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 488/975] 10.222.0.83 () {56 vars in 1101 bytes} [Wed Aug 24 11:05:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:58.460278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690601f-fb5b-476e-bd62-81944e86336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:58.461174 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690601f-fb5b-476e-bd62-81944e86336d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:58.461556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690601f-fb5b-476e-bd62-81944e86336d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:58.462128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690601f-fb5b-476e-bd62-81944e86336d None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:58.462902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690601f-fb5b-476e-bd62-81944e86336d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:58.463310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690601f-fb5b-476e-bd62-81944e86336d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:58.464903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690601f-fb5b-476e-bd62-81944e86336d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:58.485692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 488/976] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:58.486824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c538cb7-2126-427a-85b3-aa5e8678192f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:58.606697 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c538cb7-2126-427a-85b3-aa5e8678192f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:58.647002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329834f-c35f-4e70-ada9-a85f8a0e713e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:58.647575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329834f-c35f-4e70-ada9-a85f8a0e713e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:58.647899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329834f-c35f-4e70-ada9-a85f8a0e713e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:58.648179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329834f-c35f-4e70-ada9-a85f8a0e713e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:58.650431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:58.656520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329834f-c35f-4e70-ada9-a85f8a0e713e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:58.658302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329834f-c35f-4e70-ada9-a85f8a0e713e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-2329834f-c35f-4e70-ada9-a85f8a0e713e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:05:58.670487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:58.671591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:58.672475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:05:58.672475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:05:58.672475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:05:58.672475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:05:58.672475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:05:58.672475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:05:58.672475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:05:58.672475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:05:58.672475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 489/977] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:05:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:05:58.679418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690601f-fb5b-476e-bd62-81944e86336d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:58.803660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690601f-fb5b-476e-bd62-81944e86336d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:58.806589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:58.843843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35bc82-c030-42cf-adef-307e2eea4b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:58.844619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35bc82-c030-42cf-adef-307e2eea4b3a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:58.845010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35bc82-c030-42cf-adef-307e2eea4b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:58.845534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35bc82-c030-42cf-adef-307e2eea4b3a None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:58.846555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35bc82-c030-42cf-adef-307e2eea4b3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:58.847073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35bc82-c030-42cf-adef-307e2eea4b3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:58.848756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35bc82-c030-42cf-adef-307e2eea4b3a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:58.864087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 489/978] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:58.869773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329834f-c35f-4e70-ada9-a85f8a0e713e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:58.953091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5be85-f69f-443d-969d-2092c8c0cab4 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:58.953266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5be85-f69f-443d-969d-2092c8c0cab4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:58.953673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5be85-f69f-443d-969d-2092c8c0cab4 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:58.955489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 490/979] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:58.961212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40aea-520e-4e8c-a18c-9fbbc53f5408 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:58.961522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40aea-520e-4e8c-a18c-9fbbc53f5408 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:58.961990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40aea-520e-4e8c-a18c-9fbbc53f5408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:58.982347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3fb40aea-520e-4e8c-a18c-9fbbc53f5408 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:58.984153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3fb40aea-520e-4e8c-a18c-9fbbc53f5408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:58.991210 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329834f-c35f-4e70-ada9-a85f8a0e713e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:59.030854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b84bdf-653d-4731-a705-f982ec204321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:59.031466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b84bdf-653d-4731-a705-f982ec204321 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:59.031748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b84bdf-653d-4731-a705-f982ec204321 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:59.032051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b84bdf-653d-4731-a705-f982ec204321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:59.032751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b84bdf-653d-4731-a705-f982ec204321 None admin] RBAC: Adding query filter params (role.id=437c0e6d42454149ba3b095971b8a9ee, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=f9d0b6662bf947cba853388819a933b8) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:05:59.033060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b84bdf-653d-4731-a705-f982ec204321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:59.034544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b84bdf-653d-4731-a705-f982ec204321 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:59.040613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 490/980] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:05:58 2022] GET /identity/v3/role_assignments?user.id=f9d0b6662bf947cba853388819a933b8&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=437c0e6d42454149ba3b095971b8a9ee => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:05:59.078307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 491/981] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:05:59.323368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c4266-5ba0-42df-92c3-d02fee9ee7db None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:59.323661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c4266-5ba0-42df-92c3-d02fee9ee7db None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:59.323942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c4266-5ba0-42df-92c3-d02fee9ee7db None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:59.325642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 491/982] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:05:59.331260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f285d-4c51-4bcb-85bb-1bb5f6741de3 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:59.331721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f285d-4c51-4bcb-85bb-1bb5f6741de3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:59.331797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f285d-4c51-4bcb-85bb-1bb5f6741de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:59.350325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ad0f285d-4c51-4bcb-85bb-1bb5f6741de3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:05:59.351510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ad0f285d-4c51-4bcb-85bb-1bb5f6741de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:05:59.358480 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c4266-5ba0-42df-92c3-d02fee9ee7db None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:59.427668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:59.433703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 492/983] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:05:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:05:59.476939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c4266-5ba0-42df-92c3-d02fee9ee7db None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:59.512082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2e30e-4573-41f8-847a-80601e4c2a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:59.512835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2e30e-4573-41f8-847a-80601e4c2a6e None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:59.513209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2e30e-4573-41f8-847a-80601e4c2a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:59.513577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2e30e-4573-41f8-847a-80601e4c2a6e None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:59.514422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2e30e-4573-41f8-847a-80601e4c2a6e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:59.515868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2e30e-4573-41f8-847a-80601e4c2a6e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:59.613982 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 492/984] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:05:59 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:05:59.700897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f285d-4c51-4bcb-85bb-1bb5f6741de3 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:05:59.730793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:05:59.816525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f285d-4c51-4bcb-85bb-1bb5f6741de3 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:05:59.867072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bd55f-56bc-48ec-b416-01764b1c68f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:05:59.867715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bd55f-56bc-48ec-b416-01764b1c68f1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:05:59.868159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bd55f-56bc-48ec-b416-01764b1c68f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:05:59.868407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bd55f-56bc-48ec-b416-01764b1c68f1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:05:59.869062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bd55f-56bc-48ec-b416-01764b1c68f1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:05:59.870544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bd55f-56bc-48ec-b416-01764b1c68f1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e44bd55f-56bc-48ec-b416-01764b1c68f1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:05:59.879532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:05:59.880729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:05:59.881772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:05:59.882936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 493/985] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:05:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:05:59.888436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2e30e-4573-41f8-847a-80601e4c2a6e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:00.004266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2e30e-4573-41f8-847a-80601e4c2a6e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:00.023775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:00.042470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e8d56-891b-4a6f-8da4-eef21bc0b793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:00.043074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e8d56-891b-4a6f-8da4-eef21bc0b793 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:00.043359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e8d56-891b-4a6f-8da4-eef21bc0b793 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:00.043626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e8d56-891b-4a6f-8da4-eef21bc0b793 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:00.044272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e8d56-891b-4a6f-8da4-eef21bc0b793 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:00.044570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e8d56-891b-4a6f-8da4-eef21bc0b793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:00.046036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e8d56-891b-4a6f-8da4-eef21bc0b793 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:00.056025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 493/986] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:05:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:00.057944 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bd55f-56bc-48ec-b416-01764b1c68f1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:00.148699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:00.164494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bd55f-56bc-48ec-b416-01764b1c68f1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:00.204102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092aea0-a59f-48df-84a7-3c01fb80aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:00.204731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092aea0-a59f-48df-84a7-3c01fb80aa3f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:00.205007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092aea0-a59f-48df-84a7-3c01fb80aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:00.205277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092aea0-a59f-48df-84a7-3c01fb80aa3f None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:00.205961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092aea0-a59f-48df-84a7-3c01fb80aa3f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:00.206268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092aea0-a59f-48df-84a7-3c01fb80aa3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:00.207695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092aea0-a59f-48df-84a7-3c01fb80aa3f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:00.213602 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 494/987] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:06:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:00.219528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e8d56-891b-4a6f-8da4-eef21bc0b793 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:00.338622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e8d56-891b-4a6f-8da4-eef21bc0b793 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:00.371837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac5989-f48f-4f9a-bef5-5c0fa1c77cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:00.372490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5989-f48f-4f9a-bef5-5c0fa1c77cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:00.372770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5989-f48f-4f9a-bef5-5c0fa1c77cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:00.373040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5989-f48f-4f9a-bef5-5c0fa1c77cb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:00.374312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac5989-f48f-4f9a-bef5-5c0fa1c77cb4 None admin] RBAC: Adding query filter params (interface=admin, service_id=3b68442c093e4ce2bfcd59ed3d8826f4, region_id=RegionOne) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:00.374664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac5989-f48f-4f9a-bef5-5c0fa1c77cb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:00.375972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac5989-f48f-4f9a-bef5-5c0fa1c77cb4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:00.383469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 494/988] 10.222.0.83 () {56 vars in 1249 bytes} [Wed Aug 24 11:06:00 2022] GET /identity/v3/endpoints?service_id=3b68442c093e4ce2bfcd59ed3d8826f4&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:00.388929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092aea0-a59f-48df-84a7-3c01fb80aa3f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:00.493694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092aea0-a59f-48df-84a7-3c01fb80aa3f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:00.530788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd2bda-13ec-414d-bda3-11329407ee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:00.531399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd2bda-13ec-414d-bda3-11329407ee9e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:00.531673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd2bda-13ec-414d-bda3-11329407ee9e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:00.531940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd2bda-13ec-414d-bda3-11329407ee9e None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:00.532542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd2bda-13ec-414d-bda3-11329407ee9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:00.533934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd2bda-13ec-414d-bda3-11329407ee9e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:00.546250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 495/989] 10.222.0.83 () {56 vars in 1087 bytes} [Wed Aug 24 11:06:00 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:01.050614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b1922-e4f3-4870-bcaa-af4e1f1b07c9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:01.051231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b1922-e4f3-4870-bcaa-af4e1f1b07c9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:01.051231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b1922-e4f3-4870-bcaa-af4e1f1b07c9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:01.052684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 495/990] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:01.058234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632cdd8-235b-4f12-987d-def1941fd623 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:01.058522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632cdd8-235b-4f12-987d-def1941fd623 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:01.058799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632cdd8-235b-4f12-987d-def1941fd623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:01.077386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e632cdd8-235b-4f12-987d-def1941fd623 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:01.078711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e632cdd8-235b-4f12-987d-def1941fd623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:01.087290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:01.164635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 496/991] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:01.475036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b1922-e4f3-4870-bcaa-af4e1f1b07c9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:01.539978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:01.576592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b686a4-e8fd-4df7-9290-717b387023ba None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:01.577684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b686a4-e8fd-4df7-9290-717b387023ba None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:01.577684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b686a4-e8fd-4df7-9290-717b387023ba None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:01.579360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 497/992] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:01.585148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1639a6-a3b3-4f4f-85b9-a5d4c1bff572 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:01.585435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1639a6-a3b3-4f4f-85b9-a5d4c1bff572 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:01.585699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1639a6-a3b3-4f4f-85b9-a5d4c1bff572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:01.608024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6c1639a6-a3b3-4f4f-85b9-a5d4c1bff572 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:01.609941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6c1639a6-a3b3-4f4f-85b9-a5d4c1bff572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:01.618582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b1922-e4f3-4870-bcaa-af4e1f1b07c9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:01.661707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1574beb-e999-4cb3-9f1a-860a232e0d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:01.662377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1574beb-e999-4cb3-9f1a-860a232e0d5a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:01.662648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1574beb-e999-4cb3-9f1a-860a232e0d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:01.662920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1574beb-e999-4cb3-9f1a-860a232e0d5a None admin] PATH_INFO: `/v3/services/image` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:01.663516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1574beb-e999-4cb3-9f1a-860a232e0d5a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:01.665091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1574beb-e999-4cb3-9f1a-860a232e0d5a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e1574beb-e999-4cb3-9f1a-860a232e0d5a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:01.675515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:01.676724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:01.677627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 496/993] 10.222.0.83 () {56 vars in 1105 bytes} [Wed Aug 24 11:06:01 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:01.683131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1574beb-e999-4cb3-9f1a-860a232e0d5a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:01.697863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 498/994] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:01.801104 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1574beb-e999-4cb3-9f1a-860a232e0d5a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:01.813864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:01.840056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc6a09-3969-4265-9857-8916a335500e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:01.840628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc6a09-3969-4265-9857-8916a335500e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:01.840941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc6a09-3969-4265-9857-8916a335500e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:01.841212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc6a09-3969-4265-9857-8916a335500e None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:01.841843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc6a09-3969-4265-9857-8916a335500e None admin] RBAC: Adding query filter params (name=image) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:01.842170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc6a09-3969-4265-9857-8916a335500e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:01.843562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc6a09-3969-4265-9857-8916a335500e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:01.853920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 497/995] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:01.855496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1639a6-a3b3-4f4f-85b9-a5d4c1bff572 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:01.937136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:01.976096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1639a6-a3b3-4f4f-85b9-a5d4c1bff572 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:02.008787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc6a09-3969-4265-9857-8916a335500e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:02.017269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd61cf7-e10d-45f5-94b4-a6498199b3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:02.017914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd61cf7-e10d-45f5-94b4-a6498199b3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:02.018617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd61cf7-e10d-45f5-94b4-a6498199b3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:02.018617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd61cf7-e10d-45f5-94b4-a6498199b3c4 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:02.019083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd61cf7-e10d-45f5-94b4-a6498199b3c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:02.019376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd61cf7-e10d-45f5-94b4-a6498199b3c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:02.020764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd61cf7-e10d-45f5-94b4-a6498199b3c4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:02.026960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 499/996] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:02.130669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc6a09-3969-4265-9857-8916a335500e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:02.167979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86225a78-f52d-48ce-a887-2a9344f9a163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:02.168626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86225a78-f52d-48ce-a887-2a9344f9a163 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:02.168902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86225a78-f52d-48ce-a887-2a9344f9a163 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:02.169310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86225a78-f52d-48ce-a887-2a9344f9a163 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:02.169929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86225a78-f52d-48ce-a887-2a9344f9a163 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:02.171756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86225a78-f52d-48ce-a887-2a9344f9a163 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-86225a78-f52d-48ce-a887-2a9344f9a163 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:02.181660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:02.182858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:06:02.183699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:02.188383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 498/997] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:06:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:02.189763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd61cf7-e10d-45f5-94b4-a6498199b3c4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:02.303521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd61cf7-e10d-45f5-94b4-a6498199b3c4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:02.346223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542dd1ac-10c1-4540-9985-e94fffc219ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:02.346838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dd1ac-10c1-4540-9985-e94fffc219ae None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:02.347117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dd1ac-10c1-4540-9985-e94fffc219ae None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:02.347392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dd1ac-10c1-4540-9985-e94fffc219ae None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:02.348029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dd1ac-10c1-4540-9985-e94fffc219ae None admin] RBAC: Adding query filter params (name=compute) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:02.348331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dd1ac-10c1-4540-9985-e94fffc219ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:02.349723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dd1ac-10c1-4540-9985-e94fffc219ae None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:02.351794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:02.355833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 500/998] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:06:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:02.362774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86225a78-f52d-48ce-a887-2a9344f9a163 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:02.407937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:02.475653 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86225a78-f52d-48ce-a887-2a9344f9a163 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:02.514546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc0981-1cd2-4279-be78-fa288fe73bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:02.515559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc0981-1cd2-4279-be78-fa288fe73bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:02.516071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc0981-1cd2-4279-be78-fa288fe73bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:02.516543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc0981-1cd2-4279-be78-fa288fe73bf9 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:02.517375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc0981-1cd2-4279-be78-fa288fe73bf9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:02.517904 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc0981-1cd2-4279-be78-fa288fe73bf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:02.519579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc0981-1cd2-4279-be78-fa288fe73bf9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:02.525213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 499/999] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:06:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:02.531765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542dd1ac-10c1-4540-9985-e94fffc219ae None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:02.579389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf07b5-1e43-469a-8752-a5a76e9f6ce2 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:02.579762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf07b5-1e43-469a-8752-a5a76e9f6ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:02.579943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf07b5-1e43-469a-8752-a5a76e9f6ce2 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:02.581686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 500/1000] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:06:02.586817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f1ec2-84bd-44b7-bbdd-80bdae2fbbfd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:02.587106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f1ec2-84bd-44b7-bbdd-80bdae2fbbfd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:02.587385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f1ec2-84bd-44b7-bbdd-80bdae2fbbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:02.606540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-536f1ec2-84bd-44b7-bbdd-80bdae2fbbfd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:02.608789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-536f1ec2-84bd-44b7-bbdd-80bdae2fbbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:02.644884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542dd1ac-10c1-4540-9985-e94fffc219ae None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:02.682815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 501/1001] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:06:02.683922 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07584-d86f-4fd5-9e16-cc3789746fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:02.684551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07584-d86f-4fd5-9e16-cc3789746fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:02.684833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07584-d86f-4fd5-9e16-cc3789746fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:02.685103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07584-d86f-4fd5-9e16-cc3789746fa8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:02.685823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d07584-d86f-4fd5-9e16-cc3789746fa8 None admin] RBAC: Adding query filter params (interface=public, service_id=8ff62c27564647dda57e6022f7f6be05, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:02.686742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d07584-d86f-4fd5-9e16-cc3789746fa8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:02.688330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d07584-d86f-4fd5-9e16-cc3789746fa8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:02.698426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 501/1002] 10.222.0.83 () {56 vars in 1251 bytes} [Wed Aug 24 11:06:02 2022] GET /identity/v3/endpoints?service_id=8ff62c27564647dda57e6022f7f6be05&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:02.699663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f1ec2-84bd-44b7-bbdd-80bdae2fbbfd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:02.731653 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:02.804245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f1ec2-84bd-44b7-bbdd-80bdae2fbbfd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:02.852444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623254e9-1584-467f-a531-cd707a7ea399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:02.853318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623254e9-1584-467f-a531-cd707a7ea399 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:02.853730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623254e9-1584-467f-a531-cd707a7ea399 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:02.854166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623254e9-1584-467f-a531-cd707a7ea399 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:02.855085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623254e9-1584-467f-a531-cd707a7ea399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:02.857146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623254e9-1584-467f-a531-cd707a7ea399 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:02.866418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 502/1003] 10.222.0.83 () {56 vars in 1087 bytes} [Wed Aug 24 11:06:02 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:02.957222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07584-d86f-4fd5-9e16-cc3789746fa8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:03.031154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:03.071886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07584-d86f-4fd5-9e16-cc3789746fa8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:03.110124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22abdb-e07a-4833-9768-20616661e239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:03.110772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22abdb-e07a-4833-9768-20616661e239 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:03.111057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22abdb-e07a-4833-9768-20616661e239 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:03.111334 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22abdb-e07a-4833-9768-20616661e239 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:03.111928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22abdb-e07a-4833-9768-20616661e239 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:03.113325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22abdb-e07a-4833-9768-20616661e239 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-fa22abdb-e07a-4833-9768-20616661e239 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:03.122478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:03.124040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:03.124919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 502/1004] 10.222.0.83 () {56 vars in 1126 bytes} [Wed Aug 24 11:06:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:03.130609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623254e9-1584-467f-a531-cd707a7ea399 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:03.264982 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:03.272435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623254e9-1584-467f-a531-cd707a7ea399 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:03.311028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755a284-39aa-413d-92db-83473e08b12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:03.311662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a284-39aa-413d-92db-83473e08b12b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:03.311946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a284-39aa-413d-92db-83473e08b12b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:03.312224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a284-39aa-413d-92db-83473e08b12b None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:03.312979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755a284-39aa-413d-92db-83473e08b12b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:03.313158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755a284-39aa-413d-92db-83473e08b12b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:03.314576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755a284-39aa-413d-92db-83473e08b12b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:03.319886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 503/1005] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:06:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:03.326342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22abdb-e07a-4833-9768-20616661e239 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:03.438758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22abdb-e07a-4833-9768-20616661e239 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:03.466215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:03.475672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a050-ecf0-4e7a-8071-eff0b109c90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:03.476286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a050-ecf0-4e7a-8071-eff0b109c90d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:03.480725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a050-ecf0-4e7a-8071-eff0b109c90d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:03.480725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a050-ecf0-4e7a-8071-eff0b109c90d None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:03.480725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09a050-ecf0-4e7a-8071-eff0b109c90d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:03.480725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09a050-ecf0-4e7a-8071-eff0b109c90d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:03.480725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09a050-ecf0-4e7a-8071-eff0b109c90d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:03.489519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 503/1006] 10.222.0.83 () {56 vars in 1122 bytes} [Wed Aug 24 11:06:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:03.491185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755a284-39aa-413d-92db-83473e08b12b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:03.604647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755a284-39aa-413d-92db-83473e08b12b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:03.642003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2142b-133a-4a5d-9247-215a732de018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:03.642752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2142b-133a-4a5d-9247-215a732de018 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:03.643083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2142b-133a-4a5d-9247-215a732de018 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:03.643361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2142b-133a-4a5d-9247-215a732de018 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:03.644035 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2142b-133a-4a5d-9247-215a732de018 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:03.645392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2142b-133a-4a5d-9247-215a732de018 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:03.730864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 504/1007] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:06:04.006719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89055891-3052-4cb7-af02-6cedca7e686b None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:04.007256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89055891-3052-4cb7-af02-6cedca7e686b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:04.007256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89055891-3052-4cb7-af02-6cedca7e686b None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:04.008552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 504/1008] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:04.014254 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356b981-c141-4d82-bd0c-d3526b22b351 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:04.014551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356b981-c141-4d82-bd0c-d3526b22b351 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:04.014820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356b981-c141-4d82-bd0c-d3526b22b351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:04.033803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b356b981-c141-4d82-bd0c-d3526b22b351 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:04.035020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b356b981-c141-4d82-bd0c-d3526b22b351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:04.053203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:04.114052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 505/1009] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:04.379734 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89055891-3052-4cb7-af02-6cedca7e686b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:04.487351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89055891-3052-4cb7-af02-6cedca7e686b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:04.524389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb9038-3218-4024-b2eb-3a6dcf0f7a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:04.524997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb9038-3218-4024-b2eb-3a6dcf0f7a2f None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:04.525270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb9038-3218-4024-b2eb-3a6dcf0f7a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:04.525553 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb9038-3218-4024-b2eb-3a6dcf0f7a2f None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:04.526234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb9038-3218-4024-b2eb-3a6dcf0f7a2f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:04.527604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb9038-3218-4024-b2eb-3a6dcf0f7a2f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:04.639096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 505/1010] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:06:04 2022] POST /identity/v3/services => generated 242 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:04.839408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a49cc-dbe8-418c-b434-d3a545413fdd None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:04.839920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a49cc-dbe8-418c-b434-d3a545413fdd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:04.839997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a49cc-dbe8-418c-b434-d3a545413fdd None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:04.841463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 506/1011] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:04.846965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009b33-432d-4304-ad4b-8cd7b8470603 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:04.847243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009b33-432d-4304-ad4b-8cd7b8470603 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:04.847520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009b33-432d-4304-ad4b-8cd7b8470603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:04.867133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-21009b33-432d-4304-ad4b-8cd7b8470603 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:04.868373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-21009b33-432d-4304-ad4b-8cd7b8470603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:04.932943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:04.969947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 506/1012] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:04 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:05.238591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a49cc-dbe8-418c-b434-d3a545413fdd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:05.354972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a49cc-dbe8-418c-b434-d3a545413fdd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:05.391891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ac8f1-82fc-438b-a4fa-c800e5e3a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:05.392525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ac8f1-82fc-438b-a4fa-c800e5e3a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:05.392804 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ac8f1-82fc-438b-a4fa-c800e5e3a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:05.393079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ac8f1-82fc-438b-a4fa-c800e5e3a8f6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:05.393768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ac8f1-82fc-438b-a4fa-c800e5e3a8f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:05.395073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ac8f1-82fc-438b-a4fa-c800e5e3a8f6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-d75ac8f1-82fc-438b-a4fa-c800e5e3a8f6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:05.404028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:05.405135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:06:05.406567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:05.413373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 507/1013] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:06:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:05.413523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21009b33-432d-4304-ad4b-8cd7b8470603 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:05.515860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21009b33-432d-4304-ad4b-8cd7b8470603 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:05.551624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85376f93-55bd-4be4-8dac-76c40a4c7fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:05.552239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85376f93-55bd-4be4-8dac-76c40a4c7fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:05.552522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85376f93-55bd-4be4-8dac-76c40a4c7fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:05.552799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85376f93-55bd-4be4-8dac-76c40a4c7fa1 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:05.553449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85376f93-55bd-4be4-8dac-76c40a4c7fa1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:05.553743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85376f93-55bd-4be4-8dac-76c40a4c7fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:05.555179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85376f93-55bd-4be4-8dac-76c40a4c7fa1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:05.560635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 507/1014] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:06:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:05.566644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ac8f1-82fc-438b-a4fa-c800e5e3a8f6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:05.580502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:05.675571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ac8f1-82fc-438b-a4fa-c800e5e3a8f6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:05.713188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd687f1b-1211-4315-a58c-1e67c21d5896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:05.713978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd687f1b-1211-4315-a58c-1e67c21d5896 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:05.714166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd687f1b-1211-4315-a58c-1e67c21d5896 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:05.714390 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd687f1b-1211-4315-a58c-1e67c21d5896 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:05.715178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd687f1b-1211-4315-a58c-1e67c21d5896 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:05.715483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd687f1b-1211-4315-a58c-1e67c21d5896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:05.716864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd687f1b-1211-4315-a58c-1e67c21d5896 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:05.722923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 508/1015] 10.222.0.83 () {56 vars in 1112 bytes} [Wed Aug 24 11:06:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:05.728473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85376f93-55bd-4be4-8dac-76c40a4c7fa1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:05.741804 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67330b35-e92b-4365-b319-70d0942a4fca None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:05.742085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67330b35-e92b-4365-b319-70d0942a4fca None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:05.742357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67330b35-e92b-4365-b319-70d0942a4fca None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:05.743995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 509/1016] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:06:05.748932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dfa88-3ab1-46fc-8d90-04e3a5444a10 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:05.749225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dfa88-3ab1-46fc-8d90-04e3a5444a10 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:05.749600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dfa88-3ab1-46fc-8d90-04e3a5444a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:05.770083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8f8dfa88-3ab1-46fc-8d90-04e3a5444a10 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:05.771146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8f8dfa88-3ab1-46fc-8d90-04e3a5444a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:05.777962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:05.839955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85376f93-55bd-4be4-8dac-76c40a4c7fa1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:05.847110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 510/1017] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:05 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:06:05.878290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b062375-1a43-4916-90eb-86eef7de4cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:05.878901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b062375-1a43-4916-90eb-86eef7de4cd7 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:05.879176 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b062375-1a43-4916-90eb-86eef7de4cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:05.879449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b062375-1a43-4916-90eb-86eef7de4cd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:05.880137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b062375-1a43-4916-90eb-86eef7de4cd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:05.881546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b062375-1a43-4916-90eb-86eef7de4cd7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:05.965907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 508/1018] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:06.126239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dfa88-3ab1-46fc-8d90-04e3a5444a10 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:06.129462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:06.236600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dfa88-3ab1-46fc-8d90-04e3a5444a10 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:06.279541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eceecad-dbcf-4daf-9460-d1d51c8d48bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:06.283869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eceecad-dbcf-4daf-9460-d1d51c8d48bc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:06.283869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eceecad-dbcf-4daf-9460-d1d51c8d48bc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:06.283869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eceecad-dbcf-4daf-9460-d1d51c8d48bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:06.286732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eceecad-dbcf-4daf-9460-d1d51c8d48bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:06.288331 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eceecad-dbcf-4daf-9460-d1d51c8d48bc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:06.290334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 511/1019] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:06.296682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b062375-1a43-4916-90eb-86eef7de4cd7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:06.416653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:06.421866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b062375-1a43-4916-90eb-86eef7de4cd7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:06.462980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819acf6e-aad2-4d58-a5c4-15ddd3979c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:06.463602 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819acf6e-aad2-4d58-a5c4-15ddd3979c90 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:06.463882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819acf6e-aad2-4d58-a5c4-15ddd3979c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:06.464153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819acf6e-aad2-4d58-a5c4-15ddd3979c90 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:06.464829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819acf6e-aad2-4d58-a5c4-15ddd3979c90 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:06.466307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819acf6e-aad2-4d58-a5c4-15ddd3979c90 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:06.549000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 509/1020] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:06:06 2022] POST /identity/v3/projects => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:06.614833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6e5c1-ca44-4233-8464-965efdf5daac None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:06.615112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6e5c1-ca44-4233-8464-965efdf5daac None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:06.615385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6e5c1-ca44-4233-8464-965efdf5daac None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:06.616810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 512/1021] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:06.623272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ba5f2-6eaa-4ffe-8609-1e3c72fbea08 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:06.623624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ba5f2-6eaa-4ffe-8609-1e3c72fbea08 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:06.623843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ba5f2-6eaa-4ffe-8609-1e3c72fbea08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:06.644455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2c5ba5f2-6eaa-4ffe-8609-1e3c72fbea08 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:06.645684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2c5ba5f2-6eaa-4ffe-8609-1e3c72fbea08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:06.726613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 510/1022] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:06.992634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6e5c1-ca44-4233-8464-965efdf5daac None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:07.070717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:07.107001 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6e5c1-ca44-4233-8464-965efdf5daac None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:07.145648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253d42d-bf7f-4205-a02c-889a8690d1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:07.146309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253d42d-bf7f-4205-a02c-889a8690d1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:07.146589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253d42d-bf7f-4205-a02c-889a8690d1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:07.146862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253d42d-bf7f-4205-a02c-889a8690d1b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:07.147452 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253d42d-bf7f-4205-a02c-889a8690d1b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:07.148820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253d42d-bf7f-4205-a02c-889a8690d1b6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-f253d42d-bf7f-4205-a02c-889a8690d1b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:07.158631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:07.162190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:07.163058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 513/1023] 10.222.0.83 () {56 vars in 1105 bytes} [Wed Aug 24 11:06:06 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:07.166110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ba5f2-6eaa-4ffe-8609-1e3c72fbea08 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:07.256543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:07.276316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ba5f2-6eaa-4ffe-8609-1e3c72fbea08 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:07.316025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dada70f-bdb0-4654-9c25-fa083ffcb917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:07.316680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dada70f-bdb0-4654-9c25-fa083ffcb917 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:07.316960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dada70f-bdb0-4654-9c25-fa083ffcb917 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:07.317235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dada70f-bdb0-4654-9c25-fa083ffcb917 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:07.317889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dada70f-bdb0-4654-9c25-fa083ffcb917 None admin] RBAC: Adding query filter params (name=image) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:07.318204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dada70f-bdb0-4654-9c25-fa083ffcb917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:07.319586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dada70f-bdb0-4654-9c25-fa083ffcb917 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:07.334152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 511/1024] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:07.335288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253d42d-bf7f-4205-a02c-889a8690d1b6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:07.443656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253d42d-bf7f-4205-a02c-889a8690d1b6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:07.473346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:07.479118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a4a46-b66e-4c6d-86ef-8209cdcd6141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:07.479729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a4a46-b66e-4c6d-86ef-8209cdcd6141 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:07.480008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a4a46-b66e-4c6d-86ef-8209cdcd6141 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:07.480283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a4a46-b66e-4c6d-86ef-8209cdcd6141 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:07.480971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a4a46-b66e-4c6d-86ef-8209cdcd6141 None admin] RBAC: Adding query filter params (type=image) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:07.487431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a4a46-b66e-4c6d-86ef-8209cdcd6141 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:07.487431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a4a46-b66e-4c6d-86ef-8209cdcd6141 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:07.493436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 514/1025] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:07.494517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dada70f-bdb0-4654-9c25-fa083ffcb917 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:07.604132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dada70f-bdb0-4654-9c25-fa083ffcb917 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:07.639187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d181c4-a8ba-4267-aa68-4bf297558212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:07.639801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d181c4-a8ba-4267-aa68-4bf297558212 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:07.640081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d181c4-a8ba-4267-aa68-4bf297558212 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:07.640351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d181c4-a8ba-4267-aa68-4bf297558212 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:07.641063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d181c4-a8ba-4267-aa68-4bf297558212 None admin] RBAC: Adding query filter params (interface=public, service_id=845949001de44009a19db7c1776a9cdb, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:07.641432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d181c4-a8ba-4267-aa68-4bf297558212 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:07.642792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d181c4-a8ba-4267-aa68-4bf297558212 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:07.652896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 512/1026] 10.222.0.83 () {56 vars in 1251 bytes} [Wed Aug 24 11:06:07 2022] GET /identity/v3/endpoints?service_id=845949001de44009a19db7c1776a9cdb&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:07.654387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a4a46-b66e-4c6d-86ef-8209cdcd6141 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:07.764055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a4a46-b66e-4c6d-86ef-8209cdcd6141 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:07.799419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17f890-9e58-48b5-a64f-5ce6096b8bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:07.800017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17f890-9e58-48b5-a64f-5ce6096b8bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:07.800292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17f890-9e58-48b5-a64f-5ce6096b8bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:07.800561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17f890-9e58-48b5-a64f-5ce6096b8bf3 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:07.801161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17f890-9e58-48b5-a64f-5ce6096b8bf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:07.802550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17f890-9e58-48b5-a64f-5ce6096b8bf3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:07.817903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 515/1027] 10.222.0.83 () {56 vars in 1087 bytes} [Wed Aug 24 11:06:07 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:06:08.032235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc0866-0d68-495c-9b2d-e6bf7becc724 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:08.032536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc0866-0d68-495c-9b2d-e6bf7becc724 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:08.032705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc0866-0d68-495c-9b2d-e6bf7becc724 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:08.034290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 513/1028] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:08.039732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc661c-fab8-4160-ae73-63464a52f786 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:08.040193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc661c-fab8-4160-ae73-63464a52f786 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:08.040270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc661c-fab8-4160-ae73-63464a52f786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:08.060772 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b0fc661c-fab8-4160-ae73-63464a52f786 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:08.062034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b0fc661c-fab8-4160-ae73-63464a52f786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:08.129847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:08.142636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 516/1029] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:08.413123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc0866-0d68-495c-9b2d-e6bf7becc724 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:08.426593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:08.533268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc0866-0d68-495c-9b2d-e6bf7becc724 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:08.552642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e13cc-1c29-4654-83ad-a758447131b2 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:08.553010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e13cc-1c29-4654-83ad-a758447131b2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:08.553172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e13cc-1c29-4654-83ad-a758447131b2 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:08.554764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 517/1030] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:08.560609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14764910-b746-42bd-a634-2e79d412079c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:08.560935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14764910-b746-42bd-a634-2e79d412079c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:08.561199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14764910-b746-42bd-a634-2e79d412079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:08.576107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da9e30-daca-4447-bca6-48f715e476f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:08.576729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da9e30-daca-4447-bca6-48f715e476f7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:08.576968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da9e30-daca-4447-bca6-48f715e476f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:08.577236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da9e30-daca-4447-bca6-48f715e476f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:08.579473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-14764910-b746-42bd-a634-2e79d412079c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:08.580695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-14764910-b746-42bd-a634-2e79d412079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:08.589222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da9e30-daca-4447-bca6-48f715e476f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-49da9e30-daca-4447-bca6-48f715e476f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:06:08.594937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:08.595919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:08.596795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 514/1031] 10.222.0.83 () {56 vars in 1120 bytes} [Wed Aug 24 11:06:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:08.602316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da9e30-daca-4447-bca6-48f715e476f7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:08.664277 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 518/1032] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:08.680104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:08.718540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da9e30-daca-4447-bca6-48f715e476f7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:08.769073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c830063-3035-4cbd-abc8-7219d9fe3bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:08.769739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c830063-3035-4cbd-abc8-7219d9fe3bea None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:08.770072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c830063-3035-4cbd-abc8-7219d9fe3bea None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:08.770487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c830063-3035-4cbd-abc8-7219d9fe3bea None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:08.771510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c830063-3035-4cbd-abc8-7219d9fe3bea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:08.771967 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c830063-3035-4cbd-abc8-7219d9fe3bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:08.774165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c830063-3035-4cbd-abc8-7219d9fe3bea None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:08.786712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 515/1033] 10.222.0.83 () {56 vars in 1115 bytes} [Wed Aug 24 11:06:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:08.792712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14764910-b746-42bd-a634-2e79d412079c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:08.912172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14764910-b746-42bd-a634-2e79d412079c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:08.942255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:08.944180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c830063-3035-4cbd-abc8-7219d9fe3bea None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:08.962870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ded1-fab7-45d4-85b8-97cf21e107c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:08.964198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ded1-fab7-45d4-85b8-97cf21e107c8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:08.964680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ded1-fab7-45d4-85b8-97cf21e107c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:08.965438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ded1-fab7-45d4-85b8-97cf21e107c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:08.980168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640ded1-fab7-45d4-85b8-97cf21e107c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:08.981852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640ded1-fab7-45d4-85b8-97cf21e107c8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-6640ded1-fab7-45d4-85b8-97cf21e107c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:06:08.997317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:09.003980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:06:09.005141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:06:09.005141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:09.005141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:09.005141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 519/1034] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:09.007591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ded1-fab7-45d4-85b8-97cf21e107c8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:09.079642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c830063-3035-4cbd-abc8-7219d9fe3bea None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:09.122752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:09.131743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e91ae-547f-4de7-8225-ff7fdef3b9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:09.132437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e91ae-547f-4de7-8225-ff7fdef3b9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:09.132719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e91ae-547f-4de7-8225-ff7fdef3b9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:09.133036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e91ae-547f-4de7-8225-ff7fdef3b9f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:09.134515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e91ae-547f-4de7-8225-ff7fdef3b9f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:09.136084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e91ae-547f-4de7-8225-ff7fdef3b9f1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:09.137706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ded1-fab7-45d4-85b8-97cf21e107c8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:09.138194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 516/1035] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:09.145055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e91ae-547f-4de7-8225-ff7fdef3b9f1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:09.188460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d0ba9-926d-4d73-ac4f-2b2c9e591dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:09.189157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d0ba9-926d-4d73-ac4f-2b2c9e591dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:09.189443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d0ba9-926d-4d73-ac4f-2b2c9e591dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:09.189713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d0ba9-926d-4d73-ac4f-2b2c9e591dc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:09.190403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d0ba9-926d-4d73-ac4f-2b2c9e591dc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:09.190710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d0ba9-926d-4d73-ac4f-2b2c9e591dc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:09.192091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d0ba9-926d-4d73-ac4f-2b2c9e591dc6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:09.206211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 520/1036] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:09.212958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d0ba9-926d-4d73-ac4f-2b2c9e591dc6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:09.249766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:09.267662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e91ae-547f-4de7-8225-ff7fdef3b9f1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:09.313510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac9f9a-fb8a-481b-99bb-965a86c05be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:09.314423 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac9f9a-fb8a-481b-99bb-965a86c05be5 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:09.314853 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac9f9a-fb8a-481b-99bb-965a86c05be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:09.315271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac9f9a-fb8a-481b-99bb-965a86c05be5 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:09.316573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac9f9a-fb8a-481b-99bb-965a86c05be5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:09.318521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac9f9a-fb8a-481b-99bb-965a86c05be5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:09.333884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d0ba9-926d-4d73-ac4f-2b2c9e591dc6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:09.372511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79254720-53c5-457e-9e1d-2f19c49e212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:09.373171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79254720-53c5-457e-9e1d-2f19c49e212e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:09.373467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79254720-53c5-457e-9e1d-2f19c49e212e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:09.373737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79254720-53c5-457e-9e1d-2f19c49e212e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:09.381470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79254720-53c5-457e-9e1d-2f19c49e212e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:09.382962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79254720-53c5-457e-9e1d-2f19c49e212e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:09.387345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 517/1037] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:06:09 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-79254720-53c5-457e-9e1d-2f19c49e212e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:06:09.396179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:09.397398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:09.398424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 521/1038] 10.222.0.83 () {56 vars in 1093 bytes} [Wed Aug 24 11:06:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:09.410027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac9f9a-fb8a-481b-99bb-965a86c05be5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:09.480418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:09.536458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac9f9a-fb8a-481b-99bb-965a86c05be5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:09.581631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301ac07-521c-49e9-b169-59ccab221f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:09.582311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301ac07-521c-49e9-b169-59ccab221f84 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:09.582600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301ac07-521c-49e9-b169-59ccab221f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:09.582879 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301ac07-521c-49e9-b169-59ccab221f84 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:09.583658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301ac07-521c-49e9-b169-59ccab221f84 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:09.583963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301ac07-521c-49e9-b169-59ccab221f84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:09.585553 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301ac07-521c-49e9-b169-59ccab221f84 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:09.608994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 518/1039] 10.222.0.83 () {56 vars in 1133 bytes} [Wed Aug 24 11:06:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:09.614988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79254720-53c5-457e-9e1d-2f19c49e212e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:09.716853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:09.725910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79254720-53c5-457e-9e1d-2f19c49e212e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:09.760336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8e09f-3b07-4d48-bfcd-aaeb5bf222fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:09.760857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8e09f-3b07-4d48-bfcd-aaeb5bf222fd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:09.761136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8e09f-3b07-4d48-bfcd-aaeb5bf222fd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:09.761409 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8e09f-3b07-4d48-bfcd-aaeb5bf222fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:09.788553 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa794d41-0d85-4625-a8c8-d9e0c539bb88 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:09.788840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa794d41-0d85-4625-a8c8-d9e0c539bb88 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:09.789121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa794d41-0d85-4625-a8c8-d9e0c539bb88 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:09.791225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 519/1040] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:06:09.796360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b19b6-7aba-4cfd-94ac-624d11e0b1b1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:09.796668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b19b6-7aba-4cfd-94ac-624d11e0b1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:09.796950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b19b6-7aba-4cfd-94ac-624d11e0b1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:09.816245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-141b19b6-7aba-4cfd-94ac-624d11e0b1b1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:09.817449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-141b19b6-7aba-4cfd-94ac-624d11e0b1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:09.839014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8e09f-3b07-4d48-bfcd-aaeb5bf222fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:09.839770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8e09f-3b07-4d48-bfcd-aaeb5bf222fd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:09.894296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 520/1041] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:06:09.950334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:09.964891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 522/1042] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:06:09.978485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b19b6-7aba-4cfd-94ac-624d11e0b1b1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:10.024050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:10.086305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b19b6-7aba-4cfd-94ac-624d11e0b1b1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:10.121255 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c01412-8961-4837-9001-ad580b4798bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:10.121917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c01412-8961-4837-9001-ad580b4798bb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:10.122194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c01412-8961-4837-9001-ad580b4798bb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:10.122467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c01412-8961-4837-9001-ad580b4798bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:10.134477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c01412-8961-4837-9001-ad580b4798bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:10.135953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c01412-8961-4837-9001-ad580b4798bb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-c7c01412-8961-4837-9001-ad580b4798bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:06:10.152847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:10.154012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:06:10.155479 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:06:10.155479 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:10.155479 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:10.158290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 521/1043] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:10.159715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8e09f-3b07-4d48-bfcd-aaeb5bf222fd None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:10.171569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c01412-8961-4837-9001-ad580b4798bb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:10.273474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8e09f-3b07-4d48-bfcd-aaeb5bf222fd None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:10.280693 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:10.290767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c01412-8961-4837-9001-ad580b4798bb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:10.317762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433438c1-ff7e-4826-a901-bc41bf1486e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:10.318414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433438c1-ff7e-4826-a901-bc41bf1486e8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:10.318698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433438c1-ff7e-4826-a901-bc41bf1486e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:10.318965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433438c1-ff7e-4826-a901-bc41bf1486e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:10.319609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433438c1-ff7e-4826-a901-bc41bf1486e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:10.319915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433438c1-ff7e-4826-a901-bc41bf1486e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:10.321263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433438c1-ff7e-4826-a901-bc41bf1486e8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:10.336568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f39872-7e2e-485f-9750-c53fedec0af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:10.337216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f39872-7e2e-485f-9750-c53fedec0af6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:10.337525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f39872-7e2e-485f-9750-c53fedec0af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:10.337799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f39872-7e2e-485f-9750-c53fedec0af6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:10.342903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 523/1044] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:10.343043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f39872-7e2e-485f-9750-c53fedec0af6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:10.343043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f39872-7e2e-485f-9750-c53fedec0af6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:10.343848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433438c1-ff7e-4826-a901-bc41bf1486e8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-27f39872-7e2e-485f-9750-c53fedec0af6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:10.351184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:10.352228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:06:10.353085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:10.357347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 522/1045] 10.222.0.83 () {56 vars in 1105 bytes} [Wed Aug 24 11:06:10 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:10.358840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f39872-7e2e-485f-9750-c53fedec0af6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:10.403481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:10.458725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433438c1-ff7e-4826-a901-bc41bf1486e8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:10.475776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f39872-7e2e-485f-9750-c53fedec0af6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:10.501053 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccfdfd-55eb-4add-ad27-40d28ad01d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:10.501703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccfdfd-55eb-4add-ad27-40d28ad01d4e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:10.501998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccfdfd-55eb-4add-ad27-40d28ad01d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:10.502269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccfdfd-55eb-4add-ad27-40d28ad01d4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:10.509519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccfdfd-55eb-4add-ad27-40d28ad01d4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:10.511086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccfdfd-55eb-4add-ad27-40d28ad01d4e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:10.517428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e87eb0-54d4-4421-a4ac-49155f5ad954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:10.518080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e87eb0-54d4-4421-a4ac-49155f5ad954 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:10.518369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e87eb0-54d4-4421-a4ac-49155f5ad954 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:10.518642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e87eb0-54d4-4421-a4ac-49155f5ad954 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:10.519282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e87eb0-54d4-4421-a4ac-49155f5ad954 None admin] RBAC: Adding query filter params (name=image) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:10.519588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e87eb0-54d4-4421-a4ac-49155f5ad954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:10.521036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e87eb0-54d4-4421-a4ac-49155f5ad954 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b6ccfdfd-55eb-4add-ad27-40d28ad01d4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:06:10.523922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:10.524819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:10.526497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:06:10.526497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:06:10.526497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:06:10.526497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:06:10.526497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:06:10.526497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:06:10.526497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:06:10.526497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:10.526497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 524/1046] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:10.531490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccfdfd-55eb-4add-ad27-40d28ad01d4e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:10.535432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 523/1047] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:10.536025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e87eb0-54d4-4421-a4ac-49155f5ad954 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:10.562379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:10.655452 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccfdfd-55eb-4add-ad27-40d28ad01d4e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:10.655873 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e87eb0-54d4-4421-a4ac-49155f5ad954 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:10.670509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:10.695959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e789d13-29a1-43ec-971f-b310dc7c4f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:10.696380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16697f-1db3-47cc-b735-d36fb6c050e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:10.696992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16697f-1db3-47cc-b735-d36fb6c050e6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:10.697299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16697f-1db3-47cc-b735-d36fb6c050e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:10.697573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16697f-1db3-47cc-b735-d36fb6c050e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:10.697777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e789d13-29a1-43ec-971f-b310dc7c4f6d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:10.698276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e789d13-29a1-43ec-971f-b310dc7c4f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:10.698383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16697f-1db3-47cc-b735-d36fb6c050e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:10.698599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16697f-1db3-47cc-b735-d36fb6c050e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:10.699037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e789d13-29a1-43ec-971f-b310dc7c4f6d None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:10.699950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e789d13-29a1-43ec-971f-b310dc7c4f6d None admin] RBAC: Adding query filter params (type=image) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:10.700109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16697f-1db3-47cc-b735-d36fb6c050e6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:10.700576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e789d13-29a1-43ec-971f-b310dc7c4f6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:10.702925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e789d13-29a1-43ec-971f-b310dc7c4f6d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:10.713972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 524/1048] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:10.715192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e789d13-29a1-43ec-971f-b310dc7c4f6d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:10.721179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 525/1049] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:10.728423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16697f-1db3-47cc-b735-d36fb6c050e6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:10.815358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:10.830381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e789d13-29a1-43ec-971f-b310dc7c4f6d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:10.846848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16697f-1db3-47cc-b735-d36fb6c050e6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:10.869432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28009d-2ba3-4ef6-bcd4-4af6db7e2487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:10.870085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28009d-2ba3-4ef6-bcd4-4af6db7e2487 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:10.870360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28009d-2ba3-4ef6-bcd4-4af6db7e2487 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:10.870639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28009d-2ba3-4ef6-bcd4-4af6db7e2487 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:10.871304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28009d-2ba3-4ef6-bcd4-4af6db7e2487 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:10.872695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28009d-2ba3-4ef6-bcd4-4af6db7e2487 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:10.884493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fe38d-3859-4fe6-8b89-669701fa897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:10.885126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fe38d-3859-4fe6-8b89-669701fa897a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:10.885401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fe38d-3859-4fe6-8b89-669701fa897a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:10.885671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fe38d-3859-4fe6-8b89-669701fa897a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:10.886432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fe38d-3859-4fe6-8b89-669701fa897a None admin] RBAC: Adding query filter params (role.id=437c0e6d42454149ba3b095971b8a9ee, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=32a28f9fb3134f33819e53e0c72c63b4) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:10.886736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fe38d-3859-4fe6-8b89-669701fa897a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:10.888146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fe38d-3859-4fe6-8b89-669701fa897a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:10.895108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 526/1050] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:06:10 2022] GET /identity/v3/role_assignments?user.id=32a28f9fb3134f33819e53e0c72c63b4&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=437c0e6d42454149ba3b095971b8a9ee => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:10.976644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 525/1051] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:06:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:11.468526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f32654-84ed-4a71-a7b4-61d445e1c078 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:11.469034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f32654-84ed-4a71-a7b4-61d445e1c078 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:11.469111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f32654-84ed-4a71-a7b4-61d445e1c078 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:11.470466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 527/1052] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:11.475860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23183b-e6ed-492d-a2f7-bf5f2331e6e5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:11.476152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23183b-e6ed-492d-a2f7-bf5f2331e6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:11.476430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23183b-e6ed-492d-a2f7-bf5f2331e6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:11.495743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fa23183b-e6ed-492d-a2f7-bf5f2331e6e5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:11.496945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fa23183b-e6ed-492d-a2f7-bf5f2331e6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:11.576855 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 526/1053] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:11.847294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f32654-84ed-4a71-a7b4-61d445e1c078 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:11.891747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:11.959330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f32654-84ed-4a71-a7b4-61d445e1c078 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:12.003048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c04fc2-4d25-4d44-9c97-82afea8ecf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:12.003677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c04fc2-4d25-4d44-9c97-82afea8ecf1d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:12.003957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c04fc2-4d25-4d44-9c97-82afea8ecf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:12.004219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c04fc2-4d25-4d44-9c97-82afea8ecf1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:12.016164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c04fc2-4d25-4d44-9c97-82afea8ecf1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-06c04fc2-4d25-4d44-9c97-82afea8ecf1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:06:12.020898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:12.026637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:12.027538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 528/1054] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:06:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:12.031848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23183b-e6ed-492d-a2f7-bf5f2331e6e5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:12.112585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:12.148731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23183b-e6ed-492d-a2f7-bf5f2331e6e5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:12.186298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c81d5-c018-4305-8463-6e767adf59dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:12.186940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c81d5-c018-4305-8463-6e767adf59dd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:12.187219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c81d5-c018-4305-8463-6e767adf59dd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:12.187490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c81d5-c018-4305-8463-6e767adf59dd None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:12.188165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c81d5-c018-4305-8463-6e767adf59dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:12.188469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c81d5-c018-4305-8463-6e767adf59dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:12.189904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c81d5-c018-4305-8463-6e767adf59dd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:12.198931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 527/1055] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:12.205267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c04fc2-4d25-4d44-9c97-82afea8ecf1d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:12.313190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:12.319709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c04fc2-4d25-4d44-9c97-82afea8ecf1d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:12.358677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353f7ad-a21d-4ca3-b62f-05767838cb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:12.359319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353f7ad-a21d-4ca3-b62f-05767838cb96 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:12.359598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353f7ad-a21d-4ca3-b62f-05767838cb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:12.359916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353f7ad-a21d-4ca3-b62f-05767838cb96 None admin] PATH_INFO: `/v3/users/a15e916ab78648898423da7e7f3d68e7` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:12.375242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353f7ad-a21d-4ca3-b62f-05767838cb96 None admin] RBAC: Authorizing `identity:get_user(user_id=a15e916ab78648898423da7e7f3d68e7)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:12.376615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353f7ad-a21d-4ca3-b62f-05767838cb96 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:12.383232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 529/1056] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:12 2022] GET /identity/v3/users/a15e916ab78648898423da7e7f3d68e7 => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:12.384817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c81d5-c018-4305-8463-6e767adf59dd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:12.496567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c81d5-c018-4305-8463-6e767adf59dd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:12.527437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:12.534218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa64a7-ebc5-4df3-ad20-b91deda64fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:12.534837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa64a7-ebc5-4df3-ad20-b91deda64fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:12.535124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa64a7-ebc5-4df3-ad20-b91deda64fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:12.535392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa64a7-ebc5-4df3-ad20-b91deda64fb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:12.609929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa64a7-ebc5-4df3-ad20-b91deda64fb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:12.611414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa64a7-ebc5-4df3-ad20-b91deda64fb9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:12.717914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 528/1057] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:06:12.723962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353f7ad-a21d-4ca3-b62f-05767838cb96 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:12.832475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353f7ad-a21d-4ca3-b62f-05767838cb96 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:12.870992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a609d-9e7b-4d4b-9d86-c4249d7fc8f2 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:12.871260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a609d-9e7b-4d4b-9d86-c4249d7fc8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:12.871591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a609d-9e7b-4d4b-9d86-c4249d7fc8f2 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:12.873263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 529/1058] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:06:12.879374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae75bb-d095-4265-abb9-99cc6b26da0e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:12.879750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae75bb-d095-4265-abb9-99cc6b26da0e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:12.879931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae75bb-d095-4265-abb9-99cc6b26da0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:12.883243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabdc37-8d04-46aa-b0a3-61cbae8accb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:12.883857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabdc37-8d04-46aa-b0a3-61cbae8accb6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:12.884151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabdc37-8d04-46aa-b0a3-61cbae8accb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:12.884420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabdc37-8d04-46aa-b0a3-61cbae8accb6 None admin] PATH_INFO: `/v3/projects/71044b40e0134e42a473a160019ae066` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:12.885582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabdc37-8d04-46aa-b0a3-61cbae8accb6 None admin] RBAC: Authorizing `identity:get_project(project_id=71044b40e0134e42a473a160019ae066)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:12.887114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabdc37-8d04-46aa-b0a3-61cbae8accb6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:12.889539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 530/1059] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:06:12 2022] GET /identity/v3/projects/71044b40e0134e42a473a160019ae066 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:12.896179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabdc37-8d04-46aa-b0a3-61cbae8accb6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:12.901591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-edae75bb-d095-4265-abb9-99cc6b26da0e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:12.902954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-edae75bb-d095-4265-abb9-99cc6b26da0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:12.926459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:12.942031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:12.995759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 530/1060] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:06:12.997036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e43eb-3702-4c64-b795-2e26db3173ea None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:12.997252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e43eb-3702-4c64-b795-2e26db3173ea None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:12.997528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e43eb-3702-4c64-b795-2e26db3173ea None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:12.998913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 531/1061] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:13.004703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a718d-7c5a-4be2-a5cb-0ce8e7778fbd None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:13.005024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a718d-7c5a-4be2-a5cb-0ce8e7778fbd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:13.005248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a718d-7c5a-4be2-a5cb-0ce8e7778fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:13.024526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabdc37-8d04-46aa-b0a3-61cbae8accb6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:13.026670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6a1a718d-7c5a-4be2-a5cb-0ce8e7778fbd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:13.028998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6a1a718d-7c5a-4be2-a5cb-0ce8e7778fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:13.067491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2acd7-fcc2-40de-b56f-4934064e278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:13.068160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2acd7-fcc2-40de-b56f-4934064e278c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:13.068431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2acd7-fcc2-40de-b56f-4934064e278c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:13.080229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2acd7-fcc2-40de-b56f-4934064e278c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:13.080229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2acd7-fcc2-40de-b56f-4934064e278c None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=71044b40e0134e42a473a160019ae066, user.id=a15e916ab78648898423da7e7f3d68e7) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:13.080229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2acd7-fcc2-40de-b56f-4934064e278c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:13.080229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2acd7-fcc2-40de-b56f-4934064e278c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:13.080229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 531/1062] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:06:12 2022] GET /identity/v3/role_assignments?user.id=a15e916ab78648898423da7e7f3d68e7&scope.project.id=71044b40e0134e42a473a160019ae066&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:13.137904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 532/1063] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:13.289022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2acd7-fcc2-40de-b56f-4934064e278c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:13.405953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a718d-7c5a-4be2-a5cb-0ce8e7778fbd None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:13.409163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:13.430289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2acd7-fcc2-40de-b56f-4934064e278c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:13.477407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f8dc4-70ec-4582-b302-7812ca97e22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:13.478415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f8dc4-70ec-4582-b302-7812ca97e22c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:13.478975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f8dc4-70ec-4582-b302-7812ca97e22c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:13.479254 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f8dc4-70ec-4582-b302-7812ca97e22c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:13.497838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f8dc4-70ec-4582-b302-7812ca97e22c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-f36f8dc4-70ec-4582-b302-7812ca97e22c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:06:13.502726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:13.503844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:13.504766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 532/1064] 10.222.0.83 () {56 vars in 1120 bytes} [Wed Aug 24 11:06:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:13.509975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f8dc4-70ec-4582-b302-7812ca97e22c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:13.516708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:13.526521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a718d-7c5a-4be2-a5cb-0ce8e7778fbd None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:13.570475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7043d-af64-44f8-b9f9-4841a7c6c343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:13.571011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7043d-af64-44f8-b9f9-4841a7c6c343 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:13.571291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7043d-af64-44f8-b9f9-4841a7c6c343 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:13.571565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7043d-af64-44f8-b9f9-4841a7c6c343 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:13.634025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f8dc4-70ec-4582-b302-7812ca97e22c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:13.656838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7043d-af64-44f8-b9f9-4841a7c6c343 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:13.658430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7043d-af64-44f8-b9f9-4841a7c6c343 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:13.676171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27811d01-d60c-414c-ba03-b26b3de5a65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:13.677078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27811d01-d60c-414c-ba03-b26b3de5a65c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:13.677359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27811d01-d60c-414c-ba03-b26b3de5a65c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:13.677638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27811d01-d60c-414c-ba03-b26b3de5a65c None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:13.678357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27811d01-d60c-414c-ba03-b26b3de5a65c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:13.678669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27811d01-d60c-414c-ba03-b26b3de5a65c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:13.680502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27811d01-d60c-414c-ba03-b26b3de5a65c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:13.694043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 533/1065] 10.222.0.83 () {56 vars in 1115 bytes} [Wed Aug 24 11:06:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:13.698981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27811d01-d60c-414c-ba03-b26b3de5a65c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:13.712020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:13.759085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:13.773900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 533/1066] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:06:13.775042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7043d-af64-44f8-b9f9-4841a7c6c343 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:13.814607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27811d01-d60c-414c-ba03-b26b3de5a65c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:13.855179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f834bcb-81a5-4d22-9e1e-32947e5ff68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:13.856158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f834bcb-81a5-4d22-9e1e-32947e5ff68f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:13.856645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f834bcb-81a5-4d22-9e1e-32947e5ff68f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:13.857494 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f834bcb-81a5-4d22-9e1e-32947e5ff68f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:13.870124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f834bcb-81a5-4d22-9e1e-32947e5ff68f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:13.871843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f834bcb-81a5-4d22-9e1e-32947e5ff68f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:13.885307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7043d-af64-44f8-b9f9-4841a7c6c343 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0f834bcb-81a5-4d22-9e1e-32947e5ff68f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:06:13.888535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:13.889706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:06:13.895613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:06:13.895613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:13.895613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:13.895613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 534/1067] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:13.895613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f834bcb-81a5-4d22-9e1e-32947e5ff68f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:13.925968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa1d0e-2197-49c5-9f30-bfea1754f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:13.926615 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa1d0e-2197-49c5-9f30-bfea1754f4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:13.926898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa1d0e-2197-49c5-9f30-bfea1754f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:13.927172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa1d0e-2197-49c5-9f30-bfea1754f4f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:13.939460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa1d0e-2197-49c5-9f30-bfea1754f4f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:13.940997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa1d0e-2197-49c5-9f30-bfea1754f4f8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-80fa1d0e-2197-49c5-9f30-bfea1754f4f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:06:13.957293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:13.958863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:06:13.959823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:06:13.959823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:13.959823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:13.963488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 534/1068] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:13.964373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa1d0e-2197-49c5-9f30-bfea1754f4f8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:13.969750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:14.018914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f834bcb-81a5-4d22-9e1e-32947e5ff68f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:14.026054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:14.067807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98767198-7c96-4135-ae40-b25d9f3d4f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:14.068820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98767198-7c96-4135-ae40-b25d9f3d4f0c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:14.069263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98767198-7c96-4135-ae40-b25d9f3d4f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:14.069703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98767198-7c96-4135-ae40-b25d9f3d4f0c None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:14.070635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98767198-7c96-4135-ae40-b25d9f3d4f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:14.071101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98767198-7c96-4135-ae40-b25d9f3d4f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:14.072634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98767198-7c96-4135-ae40-b25d9f3d4f0c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:14.086492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 535/1069] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:14.091834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98767198-7c96-4135-ae40-b25d9f3d4f0c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:14.101657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa1d0e-2197-49c5-9f30-bfea1754f4f8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:14.139368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95c4fb-5ddc-458d-a9ca-e261c914c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:14.139893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95c4fb-5ddc-458d-a9ca-e261c914c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:14.140169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95c4fb-5ddc-458d-a9ca-e261c914c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:14.140442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95c4fb-5ddc-458d-a9ca-e261c914c8f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:14.141086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95c4fb-5ddc-458d-a9ca-e261c914c8f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:14.141385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95c4fb-5ddc-458d-a9ca-e261c914c8f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:14.142783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95c4fb-5ddc-458d-a9ca-e261c914c8f6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:14.156376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 535/1070] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:14.226548 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98767198-7c96-4135-ae40-b25d9f3d4f0c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:14.249449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:14.266424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062428b-891c-4d76-99fb-9969f593ed74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:14.267079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062428b-891c-4d76-99fb-9969f593ed74 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:14.267355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062428b-891c-4d76-99fb-9969f593ed74 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:14.267629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062428b-891c-4d76-99fb-9969f593ed74 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:14.276329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062428b-891c-4d76-99fb-9969f593ed74 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:14.277709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062428b-891c-4d76-99fb-9969f593ed74 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-f062428b-891c-4d76-99fb-9969f593ed74 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:06:14.294719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:14.296992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:14.298312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 536/1071] 10.222.0.83 () {56 vars in 1093 bytes} [Wed Aug 24 11:06:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:14.301953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95c4fb-5ddc-458d-a9ca-e261c914c8f6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:14.416265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95c4fb-5ddc-458d-a9ca-e261c914c8f6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:14.418141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:14.457628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c4a1d-5a7f-4b88-950d-67f547fccc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:14.458352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c4a1d-5a7f-4b88-950d-67f547fccc87 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:14.458635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c4a1d-5a7f-4b88-950d-67f547fccc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:14.458909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c4a1d-5a7f-4b88-950d-67f547fccc87 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:14.459657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c4a1d-5a7f-4b88-950d-67f547fccc87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:14.459965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c4a1d-5a7f-4b88-950d-67f547fccc87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:14.461420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c4a1d-5a7f-4b88-950d-67f547fccc87 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:14.480150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 536/1072] 10.222.0.83 () {56 vars in 1133 bytes} [Wed Aug 24 11:06:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:14.486738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062428b-891c-4d76-99fb-9969f593ed74 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:14.595068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062428b-891c-4d76-99fb-9969f593ed74 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:14.629005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d03e7-9864-456c-92ed-2cda13b9c42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:14.629633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d03e7-9864-456c-92ed-2cda13b9c42d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:14.629959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d03e7-9864-456c-92ed-2cda13b9c42d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:14.630233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d03e7-9864-456c-92ed-2cda13b9c42d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:14.640959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d03e7-9864-456c-92ed-2cda13b9c42d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:14.642450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d03e7-9864-456c-92ed-2cda13b9c42d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-708d03e7-9864-456c-92ed-2cda13b9c42d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:06:14.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:14.658710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:06:14.659609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:06:14.659609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:14.659609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:14.659609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 537/1073] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:14.664788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c4a1d-5a7f-4b88-950d-67f547fccc87 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:14.766211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c4a1d-5a7f-4b88-950d-67f547fccc87 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:14.800009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369f511-c5f9-4e7d-9959-3faf5f98b4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:14.800614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369f511-c5f9-4e7d-9959-3faf5f98b4ef None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:14.800884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369f511-c5f9-4e7d-9959-3faf5f98b4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:14.801151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369f511-c5f9-4e7d-9959-3faf5f98b4ef None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:14.801786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369f511-c5f9-4e7d-9959-3faf5f98b4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:14.802113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369f511-c5f9-4e7d-9959-3faf5f98b4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:14.803485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369f511-c5f9-4e7d-9959-3faf5f98b4ef None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:14.814969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 537/1074] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:14.820788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d03e7-9864-456c-92ed-2cda13b9c42d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:14.859365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:14.937471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d03e7-9864-456c-92ed-2cda13b9c42d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:14.974107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75ce30-9682-4143-be78-4db3275f1671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:14.974739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75ce30-9682-4143-be78-4db3275f1671 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:14.975020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75ce30-9682-4143-be78-4db3275f1671 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:14.975293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75ce30-9682-4143-be78-4db3275f1671 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:14.982865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75ce30-9682-4143-be78-4db3275f1671 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:14.984270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75ce30-9682-4143-be78-4db3275f1671 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-9f75ce30-9682-4143-be78-4db3275f1671 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:06:14.995791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:14.997604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:14.999691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:06:14.999691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:06:14.999691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:06:14.999691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:06:14.999691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:06:14.999691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:06:14.999691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:06:14.999691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:14.999691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 538/1075] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:15.004024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369f511-c5f9-4e7d-9959-3faf5f98b4ef None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:15.022118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4419b-df1e-4bf4-9390-32c64c6ca822 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:15.022317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4419b-df1e-4bf4-9390-32c64c6ca822 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:15.022576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4419b-df1e-4bf4-9390-32c64c6ca822 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:15.024597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 539/1076] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:06:15.030901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a67a6-86aa-49ca-88a1-b95cf2428598 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:15.031189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a67a6-86aa-49ca-88a1-b95cf2428598 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:15.031466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a67a6-86aa-49ca-88a1-b95cf2428598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:15.031574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:15.050129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-325a67a6-86aa-49ca-88a1-b95cf2428598 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:15.051355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-325a67a6-86aa-49ca-88a1-b95cf2428598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:15.118347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369f511-c5f9-4e7d-9959-3faf5f98b4ef None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:15.127541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 540/1077] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:06:15.156426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2bac1-0288-4265-bb17-06b34c8b8d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:15.157027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2bac1-0288-4265-bb17-06b34c8b8d0a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:15.157370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2bac1-0288-4265-bb17-06b34c8b8d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:15.157641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2bac1-0288-4265-bb17-06b34c8b8d0a None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:15.158384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2bac1-0288-4265-bb17-06b34c8b8d0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:15.158747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2bac1-0288-4265-bb17-06b34c8b8d0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:15.160343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2bac1-0288-4265-bb17-06b34c8b8d0a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:15.173375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 538/1078] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:15.179704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a67a6-86aa-49ca-88a1-b95cf2428598 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:15.207626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:15.291087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a67a6-86aa-49ca-88a1-b95cf2428598 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:15.327667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250e857-7b3e-40e6-9437-d07fe3912f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:15.328323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250e857-7b3e-40e6-9437-d07fe3912f27 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:15.328598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250e857-7b3e-40e6-9437-d07fe3912f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:15.328866 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250e857-7b3e-40e6-9437-d07fe3912f27 None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/32a28f9fb3134f33819e53e0c72c63b4/roles/437c0e6d42454149ba3b095971b8a9ee` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:15.331034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250e857-7b3e-40e6-9437-d07fe3912f27 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=32a28f9fb3134f33819e53e0c72c63b4, role_id=437c0e6d42454149ba3b095971b8a9ee)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:15.332439 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250e857-7b3e-40e6-9437-d07fe3912f27 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:15.406234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2bac1-0288-4265-bb17-06b34c8b8d0a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:15.417332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 541/1079] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:06:15 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/32a28f9fb3134f33819e53e0c72c63b4/roles/437c0e6d42454149ba3b095971b8a9ee => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:15.500616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:15.522809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2bac1-0288-4265-bb17-06b34c8b8d0a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:15.563119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d45658-75d0-492e-b317-7e7d9b67080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:15.563678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d45658-75d0-492e-b317-7e7d9b67080f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:15.564199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d45658-75d0-492e-b317-7e7d9b67080f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:15.564199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d45658-75d0-492e-b317-7e7d9b67080f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:15.575617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d45658-75d0-492e-b317-7e7d9b67080f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d9d45658-75d0-492e-b317-7e7d9b67080f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:06:15.580904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:15.581863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:15.582759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 539/1080] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:06:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:15.588075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250e857-7b3e-40e6-9437-d07fe3912f27 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:15.674897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:15.693384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250e857-7b3e-40e6-9437-d07fe3912f27 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:15.730175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be671e9c-65e7-41bb-95c2-6bd3314a5c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:15.730783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be671e9c-65e7-41bb-95c2-6bd3314a5c4d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:15.731059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be671e9c-65e7-41bb-95c2-6bd3314a5c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:15.731331 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be671e9c-65e7-41bb-95c2-6bd3314a5c4d None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:15.732003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be671e9c-65e7-41bb-95c2-6bd3314a5c4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:15.732395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be671e9c-65e7-41bb-95c2-6bd3314a5c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:15.733761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be671e9c-65e7-41bb-95c2-6bd3314a5c4d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:15.742669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 542/1081] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:06:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:15.748488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d45658-75d0-492e-b317-7e7d9b67080f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:15.850899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d45658-75d0-492e-b317-7e7d9b67080f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:15.885357 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fc3a4-b418-4c3c-b938-4cad8f10082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:15.885994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fc3a4-b418-4c3c-b938-4cad8f10082c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:15.886279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fc3a4-b418-4c3c-b938-4cad8f10082c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:15.886554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fc3a4-b418-4c3c-b938-4cad8f10082c None admin] PATH_INFO: `/v3/users/a15e916ab78648898423da7e7f3d68e7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:15.887712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fc3a4-b418-4c3c-b938-4cad8f10082c None admin] RBAC: Authorizing `identity:get_user(user_id=a15e916ab78648898423da7e7f3d68e7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:15.889719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fc3a4-b418-4c3c-b938-4cad8f10082c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:15.891367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 540/1082] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:15 2022] GET /identity/v3/users/a15e916ab78648898423da7e7f3d68e7 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:15.897907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be671e9c-65e7-41bb-95c2-6bd3314a5c4d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:15.999624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be671e9c-65e7-41bb-95c2-6bd3314a5c4d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:16.034266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356725-8965-4f74-9798-ef82788c2bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:16.034894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356725-8965-4f74-9798-ef82788c2bda None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:16.035168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356725-8965-4f74-9798-ef82788c2bda None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:16.035438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356725-8965-4f74-9798-ef82788c2bda None admin] PATH_INFO: `/v3/projects/71044b40e0134e42a473a160019ae066` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:16.036824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356725-8965-4f74-9798-ef82788c2bda None admin] RBAC: Authorizing `identity:get_project(project_id=71044b40e0134e42a473a160019ae066)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:16.038036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356725-8965-4f74-9798-ef82788c2bda None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:16.039917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 543/1083] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:06:15 2022] GET /identity/v3/projects/71044b40e0134e42a473a160019ae066 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:16.045694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fc3a4-b418-4c3c-b938-4cad8f10082c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:16.057375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:16.149115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fc3a4-b418-4c3c-b938-4cad8f10082c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:16.202189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e2d8f-43b0-4f66-99e0-d200a9993a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:16.202189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e2d8f-43b0-4f66-99e0-d200a9993a91 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:16.202189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e2d8f-43b0-4f66-99e0-d200a9993a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:16.202189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e2d8f-43b0-4f66-99e0-d200a9993a91 None admin] PATH_INFO: `/v3/projects/71044b40e0134e42a473a160019ae066/users/a15e916ab78648898423da7e7f3d68e7/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:16.203127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e2d8f-43b0-4f66-99e0-d200a9993a91 None admin] RBAC: Authorizing `identity:create_grant(project_id=71044b40e0134e42a473a160019ae066, user_id=a15e916ab78648898423da7e7f3d68e7, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:16.203127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e2d8f-43b0-4f66-99e0-d200a9993a91 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:16.261583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 541/1084] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:06:16 2022] PUT /identity/v3/projects/71044b40e0134e42a473a160019ae066/users/a15e916ab78648898423da7e7f3d68e7/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:17.325862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c379be-29c7-4b64-aaee-ef8fe9578175 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:17.326303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c379be-29c7-4b64-aaee-ef8fe9578175 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:17.327120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c379be-29c7-4b64-aaee-ef8fe9578175 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:17.328528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 544/1085] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:17.334222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a1f1-7cbf-46c9-b495-e7f71ada28bd None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:17.334847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a1f1-7cbf-46c9-b495-e7f71ada28bd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:17.334847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a1f1-7cbf-46c9-b495-e7f71ada28bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:17.354154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7032a1f1-7cbf-46c9-b495-e7f71ada28bd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:17.355365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7032a1f1-7cbf-46c9-b495-e7f71ada28bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:17.417768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:17.423656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 542/1086] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:17.690712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c379be-29c7-4b64-aaee-ef8fe9578175 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:17.706090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:17.796112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c379be-29c7-4b64-aaee-ef8fe9578175 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:17.832939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa062dc0-e8f4-41e4-a5e4-9ec755e00cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:17.833566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa062dc0-e8f4-41e4-a5e4-9ec755e00cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:17.833847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa062dc0-e8f4-41e4-a5e4-9ec755e00cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:17.834151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa062dc0-e8f4-41e4-a5e4-9ec755e00cc7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:17.845383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa062dc0-e8f4-41e4-a5e4-9ec755e00cc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-aa062dc0-e8f4-41e4-a5e4-9ec755e00cc7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:06:17.850130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:17.851065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:17.852315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 545/1087] 10.222.0.83 () {56 vars in 1120 bytes} [Wed Aug 24 11:06:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:17.857307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032a1f1-7cbf-46c9-b495-e7f71ada28bd None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:17.956544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032a1f1-7cbf-46c9-b495-e7f71ada28bd None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:17.991454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25babc-e0bf-40e4-abc2-21460e2d56ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:17.992057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25babc-e0bf-40e4-abc2-21460e2d56ff None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:17.992329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25babc-e0bf-40e4-abc2-21460e2d56ff None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:17.992599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25babc-e0bf-40e4-abc2-21460e2d56ff None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:17.993265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25babc-e0bf-40e4-abc2-21460e2d56ff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:17.993563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25babc-e0bf-40e4-abc2-21460e2d56ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:17.994965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25babc-e0bf-40e4-abc2-21460e2d56ff None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:18.004006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 543/1088] 10.222.0.83 () {56 vars in 1115 bytes} [Wed Aug 24 11:06:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:18.009899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa062dc0-e8f4-41e4-a5e4-9ec755e00cc7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:18.088724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:18.129590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa062dc0-e8f4-41e4-a5e4-9ec755e00cc7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:18.186150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc52c3-b6b5-428e-be8f-171eb0c08917 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:18.186435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc52c3-b6b5-428e-be8f-171eb0c08917 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:18.186744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc52c3-b6b5-428e-be8f-171eb0c08917 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:18.188138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 544/1089] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:06:18.193675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bd9a8-3dcf-43a4-ab69-df8a5e37dcdc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:18.193988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bd9a8-3dcf-43a4-ab69-df8a5e37dcdc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:18.194263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bd9a8-3dcf-43a4-ab69-df8a5e37dcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:18.212826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-be4bd9a8-3dcf-43a4-ab69-df8a5e37dcdc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:18.214003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-be4bd9a8-3dcf-43a4-ab69-df8a5e37dcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:18.258946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab85bff-a236-4c44-8c2c-775ec9b80ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:18.259590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab85bff-a236-4c44-8c2c-775ec9b80ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:18.259862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab85bff-a236-4c44-8c2c-775ec9b80ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:18.260140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab85bff-a236-4c44-8c2c-775ec9b80ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:18.271418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab85bff-a236-4c44-8c2c-775ec9b80ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:18.272972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab85bff-a236-4c44-8c2c-775ec9b80ce3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:18.282762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-5ab85bff-a236-4c44-8c2c-775ec9b80ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:06:18.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:18.289346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:06:18.290308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:06:18.290308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:18.290308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:18.290308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 546/1090] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:18.296440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab85bff-a236-4c44-8c2c-775ec9b80ce3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:18.298728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 545/1091] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:06:18.402689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab85bff-a236-4c44-8c2c-775ec9b80ce3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:18.428299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:18.439690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d41a5b-5928-46d3-9260-795efc73a4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:18.440299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d41a5b-5928-46d3-9260-795efc73a4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:18.440576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d41a5b-5928-46d3-9260-795efc73a4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:18.440852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d41a5b-5928-46d3-9260-795efc73a4e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:18.441498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d41a5b-5928-46d3-9260-795efc73a4e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:18.441793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d41a5b-5928-46d3-9260-795efc73a4e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:18.443230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d41a5b-5928-46d3-9260-795efc73a4e8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:18.455727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 547/1092] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:18.461717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bd9a8-3dcf-43a4-ab69-df8a5e37dcdc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:18.568605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d41a5b-5928-46d3-9260-795efc73a4e8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:18.581767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bd9a8-3dcf-43a4-ab69-df8a5e37dcdc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:18.622953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e6dd5-8f54-4662-a5d8-2d0b320676dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:18.623657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e6dd5-8f54-4662-a5d8-2d0b320676dd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:18.623941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e6dd5-8f54-4662-a5d8-2d0b320676dd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:18.624216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e6dd5-8f54-4662-a5d8-2d0b320676dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:18.632227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e6dd5-8f54-4662-a5d8-2d0b320676dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:18.633689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e6dd5-8f54-4662-a5d8-2d0b320676dd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d59e6dd5-8f54-4662-a5d8-2d0b320676dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:06:18.646981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:18.648138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:18.649243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 546/1093] 10.222.0.83 () {56 vars in 1093 bytes} [Wed Aug 24 11:06:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:18.653827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e6dd5-8f54-4662-a5d8-2d0b320676dd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:18.674378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d41a5b-5928-46d3-9260-795efc73a4e8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:18.706014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:18.711066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc7bb8-24dd-4902-816f-924fa94330e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:18.711696 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc7bb8-24dd-4902-816f-924fa94330e5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:18.711976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc7bb8-24dd-4902-816f-924fa94330e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:18.712249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc7bb8-24dd-4902-816f-924fa94330e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:18.723161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc7bb8-24dd-4902-816f-924fa94330e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-3ccc7bb8-24dd-4902-816f-924fa94330e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:06:18.727928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:18.728856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:18.730012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 548/1094] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:06:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:18.735554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc7bb8-24dd-4902-816f-924fa94330e5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:18.758556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e6dd5-8f54-4662-a5d8-2d0b320676dd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:18.784569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:18.794438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218cd6c-97c5-416b-9b26-3705d34f1d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:18.795043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218cd6c-97c5-416b-9b26-3705d34f1d00 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:18.795320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218cd6c-97c5-416b-9b26-3705d34f1d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:18.795595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218cd6c-97c5-416b-9b26-3705d34f1d00 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:18.796332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218cd6c-97c5-416b-9b26-3705d34f1d00 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:18.796641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218cd6c-97c5-416b-9b26-3705d34f1d00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:18.798119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218cd6c-97c5-416b-9b26-3705d34f1d00 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:18.816789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 547/1095] 10.222.0.83 () {56 vars in 1133 bytes} [Wed Aug 24 11:06:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:18.822812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218cd6c-97c5-416b-9b26-3705d34f1d00 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:18.839988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc7bb8-24dd-4902-816f-924fa94330e5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:18.875835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d408cf-309e-4d52-bc5a-0c20456c0089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:18.876510 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d408cf-309e-4d52-bc5a-0c20456c0089 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:18.876734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d408cf-309e-4d52-bc5a-0c20456c0089 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:18.877003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d408cf-309e-4d52-bc5a-0c20456c0089 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:18.877672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d408cf-309e-4d52-bc5a-0c20456c0089 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:18.878072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d408cf-309e-4d52-bc5a-0c20456c0089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:18.879382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d408cf-309e-4d52-bc5a-0c20456c0089 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:18.893748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 549/1096] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:06:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:18.895029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d408cf-309e-4d52-bc5a-0c20456c0089 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:18.936129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218cd6c-97c5-416b-9b26-3705d34f1d00 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:18.945801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:18.971180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fe324-4a7c-4dd2-a9b0-05e3b04336ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:18.971718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fe324-4a7c-4dd2-a9b0-05e3b04336ab None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:18.971992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fe324-4a7c-4dd2-a9b0-05e3b04336ab None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:18.972257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fe324-4a7c-4dd2-a9b0-05e3b04336ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:18.999233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d408cf-309e-4d52-bc5a-0c20456c0089 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:19.006369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:19.035205 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec88bd-a1e0-46cb-8c00-8da88bd3156f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:19.035832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec88bd-a1e0-46cb-8c00-8da88bd3156f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:19.036106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec88bd-a1e0-46cb-8c00-8da88bd3156f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:19.036379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec88bd-a1e0-46cb-8c00-8da88bd3156f None admin] PATH_INFO: `/v3/users/a15e916ab78648898423da7e7f3d68e7` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:19.037532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec88bd-a1e0-46cb-8c00-8da88bd3156f None admin] RBAC: Authorizing `identity:get_user(user_id=a15e916ab78648898423da7e7f3d68e7)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:19.038995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec88bd-a1e0-46cb-8c00-8da88bd3156f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:19.040574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 550/1097] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:18 2022] GET /identity/v3/users/a15e916ab78648898423da7e7f3d68e7 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:19.044467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fe324-4a7c-4dd2-a9b0-05e3b04336ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:19.046243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fe324-4a7c-4dd2-a9b0-05e3b04336ab None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:19.046693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec88bd-a1e0-46cb-8c00-8da88bd3156f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:19.175624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec88bd-a1e0-46cb-8c00-8da88bd3156f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:19.176666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 548/1098] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:06:19.183095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fe324-4a7c-4dd2-a9b0-05e3b04336ab None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:19.195032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:19.218845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af5f0c-525d-4de3-9af1-19050269bf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:19.219441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af5f0c-525d-4de3-9af1-19050269bf10 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:19.219718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af5f0c-525d-4de3-9af1-19050269bf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:19.219996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af5f0c-525d-4de3-9af1-19050269bf10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:19.265283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:19.298989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af5f0c-525d-4de3-9af1-19050269bf10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:19.300463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af5f0c-525d-4de3-9af1-19050269bf10 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:19.303266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fe324-4a7c-4dd2-a9b0-05e3b04336ab None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:19.339827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59b1a9-c7b7-42b4-a1cb-db175a8ac526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:19.340451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59b1a9-c7b7-42b4-a1cb-db175a8ac526 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:19.340727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59b1a9-c7b7-42b4-a1cb-db175a8ac526 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:19.340995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59b1a9-c7b7-42b4-a1cb-db175a8ac526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:19.351114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59b1a9-c7b7-42b4-a1cb-db175a8ac526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:19.352576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59b1a9-c7b7-42b4-a1cb-db175a8ac526 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-0b59b1a9-c7b7-42b4-a1cb-db175a8ac526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:06:19.367729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:19.368889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:06:19.369813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:06:19.369813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:06:19.369813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:19.369813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 549/1099] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:19.375242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59b1a9-c7b7-42b4-a1cb-db175a8ac526 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:19.414817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 551/1100] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:06:19.416868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af5f0c-525d-4de3-9af1-19050269bf10 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:19.443383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:19.485883 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59b1a9-c7b7-42b4-a1cb-db175a8ac526 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:19.496630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:19.521224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab01b06-d1aa-4366-9464-502a5f1a4829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:19.521836 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab01b06-d1aa-4366-9464-502a5f1a4829 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:19.522150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab01b06-d1aa-4366-9464-502a5f1a4829 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:19.522422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab01b06-d1aa-4366-9464-502a5f1a4829 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:19.522642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af5f0c-525d-4de3-9af1-19050269bf10 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:19.523070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab01b06-d1aa-4366-9464-502a5f1a4829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:19.523376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab01b06-d1aa-4366-9464-502a5f1a4829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:19.524770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab01b06-d1aa-4366-9464-502a5f1a4829 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:19.538974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 550/1101] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:19.544781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab01b06-d1aa-4366-9464-502a5f1a4829 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:19.564190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454d06-fe9b-49ac-932e-c4fbd6dcc9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:19.564883 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454d06-fe9b-49ac-932e-c4fbd6dcc9ab None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:19.565169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454d06-fe9b-49ac-932e-c4fbd6dcc9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:19.565443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454d06-fe9b-49ac-932e-c4fbd6dcc9ab None admin] PATH_INFO: `/v3/projects/71044b40e0134e42a473a160019ae066` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:19.566702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d454d06-fe9b-49ac-932e-c4fbd6dcc9ab None admin] RBAC: Authorizing `identity:get_project(project_id=71044b40e0134e42a473a160019ae066)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:19.568175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d454d06-fe9b-49ac-932e-c4fbd6dcc9ab None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:19.569861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 552/1102] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:06:19 2022] GET /identity/v3/projects/71044b40e0134e42a473a160019ae066 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:19.576421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454d06-fe9b-49ac-932e-c4fbd6dcc9ab None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:19.654972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab01b06-d1aa-4366-9464-502a5f1a4829 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:19.676285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:19.680211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454d06-fe9b-49ac-932e-c4fbd6dcc9ab None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:19.691065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d6c6-370f-4cba-bc92-e460ae2992a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:19.691749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d6c6-370f-4cba-bc92-e460ae2992a8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:19.692029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d6c6-370f-4cba-bc92-e460ae2992a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:19.692312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d6c6-370f-4cba-bc92-e460ae2992a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:19.699158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546d6c6-370f-4cba-bc92-e460ae2992a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:19.700536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546d6c6-370f-4cba-bc92-e460ae2992a8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e546d6c6-370f-4cba-bc92-e460ae2992a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:06:19.713499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:19.714666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:19.715584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:06:19.715584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:06:19.715584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:06:19.715584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:06:19.715584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:06:19.715584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:06:19.715584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:06:19.715584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:19.715584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 551/1103] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:06:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:06:19.719443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27b2ad-b550-46f6-80ac-d27b9fb2ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:19.720050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27b2ad-b550-46f6-80ac-d27b9fb2ac5a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:19.720334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27b2ad-b550-46f6-80ac-d27b9fb2ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:19.720601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27b2ad-b550-46f6-80ac-d27b9fb2ac5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:19.720696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d6c6-370f-4cba-bc92-e460ae2992a8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:19.721327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27b2ad-b550-46f6-80ac-d27b9fb2ac5a None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=71044b40e0134e42a473a160019ae066, user.id=a15e916ab78648898423da7e7f3d68e7) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:19.721629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27b2ad-b550-46f6-80ac-d27b9fb2ac5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:19.723084 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27b2ad-b550-46f6-80ac-d27b9fb2ac5a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:19.729865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 553/1104] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:06:19 2022] GET /identity/v3/role_assignments?user.id=a15e916ab78648898423da7e7f3d68e7&scope.project.id=71044b40e0134e42a473a160019ae066&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:19.833677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d6c6-370f-4cba-bc92-e460ae2992a8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:19.871401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa31483-11a6-46f9-a209-e66d8b7981e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:19.872019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa31483-11a6-46f9-a209-e66d8b7981e7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:19.872298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa31483-11a6-46f9-a209-e66d8b7981e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:19.872573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa31483-11a6-46f9-a209-e66d8b7981e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:19.873809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa31483-11a6-46f9-a209-e66d8b7981e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:19.874138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa31483-11a6-46f9-a209-e66d8b7981e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:19.875589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa31483-11a6-46f9-a209-e66d8b7981e7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:19.889630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 552/1105] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:06:19.896091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27b2ad-b550-46f6-80ac-d27b9fb2ac5a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:19.906309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:20.019905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27b2ad-b550-46f6-80ac-d27b9fb2ac5a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:20.064269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e8ca-cfd3-4e01-8191-794856a89396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:20.064880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e8ca-cfd3-4e01-8191-794856a89396 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:20.065155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e8ca-cfd3-4e01-8191-794856a89396 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:20.065424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e8ca-cfd3-4e01-8191-794856a89396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:20.066199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e8ca-cfd3-4e01-8191-794856a89396 None admin] RBAC: Adding query filter params (role.id=437c0e6d42454149ba3b095971b8a9ee, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=32a28f9fb3134f33819e53e0c72c63b4) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:20.066508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e8ca-cfd3-4e01-8191-794856a89396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:20.067943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e8ca-cfd3-4e01-8191-794856a89396 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:20.074997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 554/1106] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:06:19 2022] GET /identity/v3/role_assignments?user.id=32a28f9fb3134f33819e53e0c72c63b4&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=437c0e6d42454149ba3b095971b8a9ee => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:21.732876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d96e54-2202-44f6-99c5-3d15927cf115 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:21.733409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d96e54-2202-44f6-99c5-3d15927cf115 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:21.733409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d96e54-2202-44f6-99c5-3d15927cf115 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:21.734705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 553/1107] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:21.740052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ea918-02b5-46bc-a51a-85f83852f7f1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:21.740502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ea918-02b5-46bc-a51a-85f83852f7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:21.740604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ea918-02b5-46bc-a51a-85f83852f7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:21.758456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7a7ea918-02b5-46bc-a51a-85f83852f7f1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:21.759650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7a7ea918-02b5-46bc-a51a-85f83852f7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:21.823687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:21.828761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 555/1108] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:22.113898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d96e54-2202-44f6-99c5-3d15927cf115 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:22.161410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:22.221522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d96e54-2202-44f6-99c5-3d15927cf115 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:22.255591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c5fef-66e3-4a00-adcd-a8347065b037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:22.256208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c5fef-66e3-4a00-adcd-a8347065b037 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:22.256483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c5fef-66e3-4a00-adcd-a8347065b037 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:22.256751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c5fef-66e3-4a00-adcd-a8347065b037 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:22.257857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c5fef-66e3-4a00-adcd-a8347065b037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:22.259325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c5fef-66e3-4a00-adcd-a8347065b037 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:22.260880 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 554/1109] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:22.266727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ea918-02b5-46bc-a51a-85f83852f7f1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:22.372809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ea918-02b5-46bc-a51a-85f83852f7f1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:22.407328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6ec44-b827-4f83-bc33-b09c9b27c9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:22.407966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6ec44-b827-4f83-bc33-b09c9b27c9c0 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:22.408247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6ec44-b827-4f83-bc33-b09c9b27c9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:22.408523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6ec44-b827-4f83-bc33-b09c9b27c9c0 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:22.409300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6ec44-b827-4f83-bc33-b09c9b27c9c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:22.410766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6ec44-b827-4f83-bc33-b09c9b27c9c0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:22.506809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 556/1110] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:06:22 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:24.463800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe839-511f-4974-b3ba-7e9def9bd8fd None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:24.464368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe839-511f-4974-b3ba-7e9def9bd8fd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:24.464368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe839-511f-4974-b3ba-7e9def9bd8fd None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:24.465681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 555/1111] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:24.471734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47717d82-5e8b-4157-b27c-979a3fc19a90 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:24.472153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47717d82-5e8b-4157-b27c-979a3fc19a90 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:24.472265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47717d82-5e8b-4157-b27c-979a3fc19a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:24.490652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-47717d82-5e8b-4157-b27c-979a3fc19a90 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:24.491835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-47717d82-5e8b-4157-b27c-979a3fc19a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:24.543478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:24.559789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 557/1112] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:24.845654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fe839-511f-4974-b3ba-7e9def9bd8fd None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:24.947871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fe839-511f-4974-b3ba-7e9def9bd8fd None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:24.957910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:24.982282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c3753-0692-4605-b80a-977d589fba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:24.982912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c3753-0692-4605-b80a-977d589fba11 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:24.983208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c3753-0692-4605-b80a-977d589fba11 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:24.983494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c3753-0692-4605-b80a-977d589fba11 None admin] PATH_INFO: `/v3/roles/e08fcbbf38b344f48f71ed9ceb314293` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:24.984898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c3753-0692-4605-b80a-977d589fba11 None admin] RBAC: Authorizing `identity:get_role(role_id=e08fcbbf38b344f48f71ed9ceb314293)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:24.986322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c3753-0692-4605-b80a-977d589fba11 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:24.987652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 556/1113] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:24 2022] GET /identity/v3/roles/e08fcbbf38b344f48f71ed9ceb314293 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:24.993692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47717d82-5e8b-4157-b27c-979a3fc19a90 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:25.094979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47717d82-5e8b-4157-b27c-979a3fc19a90 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:25.129192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd61da-9d60-4631-98f6-c16466acf370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:25.129740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd61da-9d60-4631-98f6-c16466acf370 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:25.130048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd61da-9d60-4631-98f6-c16466acf370 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:25.130324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd61da-9d60-4631-98f6-c16466acf370 None admin] PATH_INFO: `/v3/users/27bddcbba052471b86419925a3be01b0` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:25.144687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd61da-9d60-4631-98f6-c16466acf370 None admin] RBAC: Authorizing `identity:get_user(user_id=27bddcbba052471b86419925a3be01b0)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:25.147447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd61da-9d60-4631-98f6-c16466acf370 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:25.150140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 558/1114] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:24 2022] GET /identity/v3/users/27bddcbba052471b86419925a3be01b0 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:25.156208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c3753-0692-4605-b80a-977d589fba11 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:25.261310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c3753-0692-4605-b80a-977d589fba11 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:25.294718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1de3a-e883-4552-8bfb-eb424c344f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:25.295321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1de3a-e883-4552-8bfb-eb424c344f87 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:25.295603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1de3a-e883-4552-8bfb-eb424c344f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:25.296499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1de3a-e883-4552-8bfb-eb424c344f87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:25.342434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:25.365267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1de3a-e883-4552-8bfb-eb424c344f87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:25.366746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1de3a-e883-4552-8bfb-eb424c344f87 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:25.469034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 557/1115] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:06:25.475050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd61da-9d60-4631-98f6-c16466acf370 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:25.521779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:25.575970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd61da-9d60-4631-98f6-c16466acf370 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:25.610414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5ef23-9ee3-4c57-9ae1-9a828c163d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:25.611037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5ef23-9ee3-4c57-9ae1-9a828c163d6f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:25.611307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5ef23-9ee3-4c57-9ae1-9a828c163d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:25.611579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5ef23-9ee3-4c57-9ae1-9a828c163d6f None admin] PATH_INFO: `/v3/projects/71044b40e0134e42a473a160019ae066` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:25.612685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5ef23-9ee3-4c57-9ae1-9a828c163d6f None admin] RBAC: Authorizing `identity:get_project(project_id=71044b40e0134e42a473a160019ae066)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:25.614139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5ef23-9ee3-4c57-9ae1-9a828c163d6f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:25.615770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 559/1116] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:06:25 2022] GET /identity/v3/projects/71044b40e0134e42a473a160019ae066 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:25.622173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1de3a-e883-4552-8bfb-eb424c344f87 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:25.699633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:25.723115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1de3a-e883-4552-8bfb-eb424c344f87 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:25.806271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351d7e0-d619-44c3-b7db-eb22a6edfebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:25.806887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351d7e0-d619-44c3-b7db-eb22a6edfebd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:25.807188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351d7e0-d619-44c3-b7db-eb22a6edfebd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:25.807552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351d7e0-d619-44c3-b7db-eb22a6edfebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:25.808280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351d7e0-d619-44c3-b7db-eb22a6edfebd None admin] RBAC: Adding query filter params (role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=71044b40e0134e42a473a160019ae066, user.id=27bddcbba052471b86419925a3be01b0) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:25.808586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351d7e0-d619-44c3-b7db-eb22a6edfebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:25.810063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351d7e0-d619-44c3-b7db-eb22a6edfebd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:25.815576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 558/1117] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:06:25 2022] GET /identity/v3/role_assignments?user.id=27bddcbba052471b86419925a3be01b0&scope.project.id=71044b40e0134e42a473a160019ae066&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:27.721748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1091c2-b45e-4115-89c1-449fe2f86b24 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:27.722302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1091c2-b45e-4115-89c1-449fe2f86b24 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:27.722302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1091c2-b45e-4115-89c1-449fe2f86b24 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:27.723590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 560/1118] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:27.728842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae09f0-a266-4223-9caf-cd043256e94e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:27.729278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae09f0-a266-4223-9caf-cd043256e94e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:27.729381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae09f0-a266-4223-9caf-cd043256e94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:27.747278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-94ae09f0-a266-4223-9caf-cd043256e94e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:27.748485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-94ae09f0-a266-4223-9caf-cd043256e94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:27.815912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 559/1119] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:28.084484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1091c2-b45e-4115-89c1-449fe2f86b24 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:28.193139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1091c2-b45e-4115-89c1-449fe2f86b24 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:28.209607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:28.230986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394d11b-01bc-463f-863e-0d05d2a7908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:28.231614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394d11b-01bc-463f-863e-0d05d2a7908e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:28.231898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394d11b-01bc-463f-863e-0d05d2a7908e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:28.232220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394d11b-01bc-463f-863e-0d05d2a7908e None admin] PATH_INFO: `/v3/roles/e08fcbbf38b344f48f71ed9ceb314293` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:28.234157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394d11b-01bc-463f-863e-0d05d2a7908e None admin] RBAC: Authorizing `identity:get_role(role_id=e08fcbbf38b344f48f71ed9ceb314293)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:28.235524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394d11b-01bc-463f-863e-0d05d2a7908e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:28.237335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 561/1120] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:28 2022] GET /identity/v3/roles/e08fcbbf38b344f48f71ed9ceb314293 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:28.242964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae09f0-a266-4223-9caf-cd043256e94e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:28.312886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:28.344063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae09f0-a266-4223-9caf-cd043256e94e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:28.379760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989bddb-2747-4c37-ac85-04fda1b4896d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:28.380394 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989bddb-2747-4c37-ac85-04fda1b4896d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:28.380672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989bddb-2747-4c37-ac85-04fda1b4896d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:28.380943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989bddb-2747-4c37-ac85-04fda1b4896d None admin] PATH_INFO: `/v3/users/27bddcbba052471b86419925a3be01b0` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:28.382182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989bddb-2747-4c37-ac85-04fda1b4896d None admin] RBAC: Authorizing `identity:get_user(user_id=27bddcbba052471b86419925a3be01b0)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:28.383630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989bddb-2747-4c37-ac85-04fda1b4896d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:28.385226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 560/1121] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:28 2022] GET /identity/v3/users/27bddcbba052471b86419925a3be01b0 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:28.391325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394d11b-01bc-463f-863e-0d05d2a7908e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:28.492321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394d11b-01bc-463f-863e-0d05d2a7908e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:28.526696 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9c892-331c-4b9b-8a88-e93672dccf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:28.527326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9c892-331c-4b9b-8a88-e93672dccf3c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:28.527604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9c892-331c-4b9b-8a88-e93672dccf3c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:28.527880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9c892-331c-4b9b-8a88-e93672dccf3c None admin] PATH_INFO: `/v3/projects/71044b40e0134e42a473a160019ae066` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:28.529004 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9c892-331c-4b9b-8a88-e93672dccf3c None admin] RBAC: Authorizing `identity:get_project(project_id=71044b40e0134e42a473a160019ae066)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:28.531248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9c892-331c-4b9b-8a88-e93672dccf3c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:28.532921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 562/1122] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:06:28 2022] GET /identity/v3/projects/71044b40e0134e42a473a160019ae066 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:28.539017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989bddb-2747-4c37-ac85-04fda1b4896d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:28.638453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989bddb-2747-4c37-ac85-04fda1b4896d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:28.672502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f7ef1-5332-4b4e-b762-9b38b84409a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:28.673156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7ef1-5332-4b4e-b762-9b38b84409a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:28.673435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7ef1-5332-4b4e-b762-9b38b84409a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:28.673710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7ef1-5332-4b4e-b762-9b38b84409a4 None admin] PATH_INFO: `/v3/projects/71044b40e0134e42a473a160019ae066/users/27bddcbba052471b86419925a3be01b0/roles/e08fcbbf38b344f48f71ed9ceb314293` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:28.675921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f7ef1-5332-4b4e-b762-9b38b84409a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=71044b40e0134e42a473a160019ae066, user_id=27bddcbba052471b86419925a3be01b0, role_id=e08fcbbf38b344f48f71ed9ceb314293)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:28.677370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f7ef1-5332-4b4e-b762-9b38b84409a4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:28.747041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:28.752432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 561/1123] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:06:28 2022] PUT /identity/v3/projects/71044b40e0134e42a473a160019ae066/users/27bddcbba052471b86419925a3be01b0/roles/e08fcbbf38b344f48f71ed9ceb314293 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:30.679450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743b69d-fd22-47d7-81f8-6f88582ae875 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:30.680044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743b69d-fd22-47d7-81f8-6f88582ae875 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:30.680044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743b69d-fd22-47d7-81f8-6f88582ae875 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:30.681627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 563/1124] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:30.687068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfb0d-54ad-4697-bc88-05e28c2317b5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:30.687512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfb0d-54ad-4697-bc88-05e28c2317b5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:30.687605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfb0d-54ad-4697-bc88-05e28c2317b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:30.706122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f4abfb0d-54ad-4697-bc88-05e28c2317b5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:30.707318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f4abfb0d-54ad-4697-bc88-05e28c2317b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:30.773721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 562/1125] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:06:31.043491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743b69d-fd22-47d7-81f8-6f88582ae875 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:31.094926 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:31.147741 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743b69d-fd22-47d7-81f8-6f88582ae875 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:31.184670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50a68c-06dc-4b74-b750-d96a9fd8bbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:31.185300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50a68c-06dc-4b74-b750-d96a9fd8bbea None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:31.185582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50a68c-06dc-4b74-b750-d96a9fd8bbea None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:31.185857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50a68c-06dc-4b74-b750-d96a9fd8bbea None admin] PATH_INFO: `/v3/roles/e08fcbbf38b344f48f71ed9ceb314293` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:31.187332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50a68c-06dc-4b74-b750-d96a9fd8bbea None admin] RBAC: Authorizing `identity:get_role(role_id=e08fcbbf38b344f48f71ed9ceb314293)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:31.188711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50a68c-06dc-4b74-b750-d96a9fd8bbea None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:31.190086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 564/1126] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:31 2022] GET /identity/v3/roles/e08fcbbf38b344f48f71ed9ceb314293 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:31.196308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abfb0d-54ad-4697-bc88-05e28c2317b5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:31.294133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abfb0d-54ad-4697-bc88-05e28c2317b5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:31.320677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:31.329087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36c160-7232-47ad-bcf4-c7e4ef7f63dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:31.337770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36c160-7232-47ad-bcf4-c7e4ef7f63dc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:31.337770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36c160-7232-47ad-bcf4-c7e4ef7f63dc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:31.337770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36c160-7232-47ad-bcf4-c7e4ef7f63dc None admin] PATH_INFO: `/v3/users/27bddcbba052471b86419925a3be01b0` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:31.337770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36c160-7232-47ad-bcf4-c7e4ef7f63dc None admin] RBAC: Authorizing `identity:get_user(user_id=27bddcbba052471b86419925a3be01b0)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:31.337770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36c160-7232-47ad-bcf4-c7e4ef7f63dc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:31.337770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 563/1127] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:31 2022] GET /identity/v3/users/27bddcbba052471b86419925a3be01b0 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:31.340295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50a68c-06dc-4b74-b750-d96a9fd8bbea None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:31.445445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50a68c-06dc-4b74-b750-d96a9fd8bbea None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:31.462003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:31.479577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6150788-0400-4a91-813c-038bc9ce9c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:31.480174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6150788-0400-4a91-813c-038bc9ce9c0d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:31.480449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6150788-0400-4a91-813c-038bc9ce9c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:31.480716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6150788-0400-4a91-813c-038bc9ce9c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:31.546781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6150788-0400-4a91-813c-038bc9ce9c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:31.548155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6150788-0400-4a91-813c-038bc9ce9c0d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:31.644814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 565/1128] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:06:31.651097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36c160-7232-47ad-bcf4-c7e4ef7f63dc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:31.759875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36c160-7232-47ad-bcf4-c7e4ef7f63dc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:31.794769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ee769-3376-4c63-aa34-72c5574dea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:31.795444 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ee769-3376-4c63-aa34-72c5574dea62 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:31.795681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ee769-3376-4c63-aa34-72c5574dea62 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:31.795952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ee769-3376-4c63-aa34-72c5574dea62 None admin] PATH_INFO: `/v3/projects/71044b40e0134e42a473a160019ae066` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:31.797126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ee769-3376-4c63-aa34-72c5574dea62 None admin] RBAC: Authorizing `identity:get_project(project_id=71044b40e0134e42a473a160019ae066)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:31.798560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ee769-3376-4c63-aa34-72c5574dea62 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:31.800198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 564/1129] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:06:31 2022] GET /identity/v3/projects/71044b40e0134e42a473a160019ae066 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:31.807000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6150788-0400-4a91-813c-038bc9ce9c0d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:31.824875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:31.905253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6150788-0400-4a91-813c-038bc9ce9c0d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:31.939964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b731b-c728-4fdc-8dcb-8d5229c60aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:31.940558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b731b-c728-4fdc-8dcb-8d5229c60aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:31.940837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b731b-c728-4fdc-8dcb-8d5229c60aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:31.941111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b731b-c728-4fdc-8dcb-8d5229c60aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:31.941851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b731b-c728-4fdc-8dcb-8d5229c60aa8 None admin] RBAC: Adding query filter params (role.id=e08fcbbf38b344f48f71ed9ceb314293, scope.project.id=71044b40e0134e42a473a160019ae066, user.id=27bddcbba052471b86419925a3be01b0) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:31.942187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b731b-c728-4fdc-8dcb-8d5229c60aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:31.943618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b731b-c728-4fdc-8dcb-8d5229c60aa8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:31.949420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 566/1130] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:06:31 2022] GET /identity/v3/role_assignments?user.id=27bddcbba052471b86419925a3be01b0&scope.project.id=71044b40e0134e42a473a160019ae066&role.id=e08fcbbf38b344f48f71ed9ceb314293 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:33.922085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470df5d-daa5-4b24-9f3d-ec8532748f93 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.922646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470df5d-daa5-4b24-9f3d-ec8532748f93 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.922646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470df5d-daa5-4b24-9f3d-ec8532748f93 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.923959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 565/1131] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:33.929420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa78d0f-0199-4d0a-bd7e-680ea5736455 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:33.929900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa78d0f-0199-4d0a-bd7e-680ea5736455 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:33.930154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa78d0f-0199-4d0a-bd7e-680ea5736455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:33.948051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-efa78d0f-0199-4d0a-bd7e-680ea5736455 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:33.949247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-efa78d0f-0199-4d0a-bd7e-680ea5736455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:34.015762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 567/1132] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:34.287700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470df5d-daa5-4b24-9f3d-ec8532748f93 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.360442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:34.399074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470df5d-daa5-4b24-9f3d-ec8532748f93 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.435820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93122ca-41e5-47a1-8e1f-91727f9a098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.436457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93122ca-41e5-47a1-8e1f-91727f9a098c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.436735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93122ca-41e5-47a1-8e1f-91727f9a098c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.437007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93122ca-41e5-47a1-8e1f-91727f9a098c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.438180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93122ca-41e5-47a1-8e1f-91727f9a098c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.439625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93122ca-41e5-47a1-8e1f-91727f9a098c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.441227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 566/1133] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:34.447158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa78d0f-0199-4d0a-bd7e-680ea5736455 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:34.459436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:34.546902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa78d0f-0199-4d0a-bd7e-680ea5736455 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:34.582014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae7680-b451-4d29-be8f-bc893f74985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:34.582553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae7680-b451-4d29-be8f-bc893f74985e None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:34.582833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae7680-b451-4d29-be8f-bc893f74985e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:34.583105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae7680-b451-4d29-be8f-bc893f74985e None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:34.583761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae7680-b451-4d29-be8f-bc893f74985e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:34.585177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae7680-b451-4d29-be8f-bc893f74985e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:34.655495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 568/1134] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:06:34 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:36.637965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdae867-245b-4ce7-8389-fc3f086a91d5 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.638467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdae867-245b-4ce7-8389-fc3f086a91d5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.638467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdae867-245b-4ce7-8389-fc3f086a91d5 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.640269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 567/1135] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:36.645593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d091-5094-405b-8bac-f593bcb4bb7a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:36.646041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d091-5094-405b-8bac-f593bcb4bb7a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:36.646169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d091-5094-405b-8bac-f593bcb4bb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:36.665469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4c98d091-5094-405b-8bac-f593bcb4bb7a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:36.666707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4c98d091-5094-405b-8bac-f593bcb4bb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:36.724044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:36.734645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 569/1136] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:37.002150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdae867-245b-4ce7-8389-fc3f086a91d5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:37.106311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdae867-245b-4ce7-8389-fc3f086a91d5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:37.135499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:37.140870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b28483-141e-4eea-b7b7-2be5d5fe50aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:37.141486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b28483-141e-4eea-b7b7-2be5d5fe50aa None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.141762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b28483-141e-4eea-b7b7-2be5d5fe50aa None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.142057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b28483-141e-4eea-b7b7-2be5d5fe50aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.143163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b28483-141e-4eea-b7b7-2be5d5fe50aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:37.144588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b28483-141e-4eea-b7b7-2be5d5fe50aa None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:37.146188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 568/1137] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:06:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:37.155775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98d091-5094-405b-8bac-f593bcb4bb7a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:37.251886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98d091-5094-405b-8bac-f593bcb4bb7a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:37.285606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42d0e5-c9ce-4118-a628-07cdd89dca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:37.286249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42d0e5-c9ce-4118-a628-07cdd89dca88 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:37.286535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42d0e5-c9ce-4118-a628-07cdd89dca88 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:37.286807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42d0e5-c9ce-4118-a628-07cdd89dca88 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:37.287467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42d0e5-c9ce-4118-a628-07cdd89dca88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:37.288908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42d0e5-c9ce-4118-a628-07cdd89dca88 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:37.368955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 570/1138] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:06:37 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:06:39.353076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1662cdf-9307-4314-b6c9-861ae92c1252 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:39.353592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1662cdf-9307-4314-b6c9-861ae92c1252 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:39.353688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1662cdf-9307-4314-b6c9-861ae92c1252 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.355000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 569/1139] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:39.360902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc121e-3232-43f9-9555-763004969fa2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:39.361347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc121e-3232-43f9-9555-763004969fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:39.361456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc121e-3232-43f9-9555-763004969fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.379127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f0dc121e-3232-43f9-9555-763004969fa2 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:39.380321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f0dc121e-3232-43f9-9555-763004969fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:39.431401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:39.454718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 571/1140] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:06:39.719271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1662cdf-9307-4314-b6c9-861ae92c1252 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:39.819257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1662cdf-9307-4314-b6c9-861ae92c1252 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:39.854185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b76ad9-47a1-43b3-8f43-5324f1f7317e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:39.854822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b76ad9-47a1-43b3-8f43-5324f1f7317e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:39.855100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b76ad9-47a1-43b3-8f43-5324f1f7317e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:39.855370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b76ad9-47a1-43b3-8f43-5324f1f7317e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:39.866343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b76ad9-47a1-43b3-8f43-5324f1f7317e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-c8b76ad9-47a1-43b3-8f43-5324f1f7317e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:06:39.871101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:39.872319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:39.874047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 570/1141] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:39.878792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc121e-3232-43f9-9555-763004969fa2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:39.988674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc121e-3232-43f9-9555-763004969fa2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:40.022635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e49d5-2181-4a9d-8d32-e714f24b05a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:40.023261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e49d5-2181-4a9d-8d32-e714f24b05a1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:40.023530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e49d5-2181-4a9d-8d32-e714f24b05a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:40.023797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e49d5-2181-4a9d-8d32-e714f24b05a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:40.024459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e49d5-2181-4a9d-8d32-e714f24b05a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:40.024761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e49d5-2181-4a9d-8d32-e714f24b05a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:40.026209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e49d5-2181-4a9d-8d32-e714f24b05a1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:40.034715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 572/1142] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:40.040692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b76ad9-47a1-43b3-8f43-5324f1f7317e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:40.101783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:40.143077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b76ad9-47a1-43b3-8f43-5324f1f7317e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:40.178951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8c4bc-9467-4bd8-b27d-6a99a56fa8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:40.179591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8c4bc-9467-4bd8-b27d-6a99a56fa8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:40.179871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8c4bc-9467-4bd8-b27d-6a99a56fa8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:40.180141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8c4bc-9467-4bd8-b27d-6a99a56fa8d1 None admin] PATH_INFO: `/v3/users/5fcdf00e89ce4ea492ab5c4e20a65d76` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:40.200270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8c4bc-9467-4bd8-b27d-6a99a56fa8d1 None admin] RBAC: Authorizing `identity:get_user(user_id=5fcdf00e89ce4ea492ab5c4e20a65d76)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:40.202120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8c4bc-9467-4bd8-b27d-6a99a56fa8d1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:40.204080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 571/1143] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:40 2022] GET /identity/v3/users/5fcdf00e89ce4ea492ab5c4e20a65d76 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:40.210497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e49d5-2181-4a9d-8d32-e714f24b05a1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:40.267584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:40.329151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e49d5-2181-4a9d-8d32-e714f24b05a1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:40.371630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565de56-c350-49eb-95d9-a37758878dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:40.372299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565de56-c350-49eb-95d9-a37758878dfc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:40.372487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565de56-c350-49eb-95d9-a37758878dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:40.372755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565de56-c350-49eb-95d9-a37758878dfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:40.442590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565de56-c350-49eb-95d9-a37758878dfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:40.444028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565de56-c350-49eb-95d9-a37758878dfc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:40.483716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:40.543038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 573/1144] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:06:40.549645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8c4bc-9467-4bd8-b27d-6a99a56fa8d1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:40.650760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8c4bc-9467-4bd8-b27d-6a99a56fa8d1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:40.668622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:40.686495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864f234-abc2-4341-a732-8fe7b332d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:40.692476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864f234-abc2-4341-a732-8fe7b332d1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:40.692476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864f234-abc2-4341-a732-8fe7b332d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:40.692476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864f234-abc2-4341-a732-8fe7b332d1d3 None admin] PATH_INFO: `/v3/projects/367396f254c142b8ad02553e42e76250` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:40.692476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864f234-abc2-4341-a732-8fe7b332d1d3 None admin] RBAC: Authorizing `identity:get_project(project_id=367396f254c142b8ad02553e42e76250)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:40.692476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864f234-abc2-4341-a732-8fe7b332d1d3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:40.692476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 572/1145] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:06:40 2022] GET /identity/v3/projects/367396f254c142b8ad02553e42e76250 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:40.698293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565de56-c350-49eb-95d9-a37758878dfc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:40.795032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565de56-c350-49eb-95d9-a37758878dfc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:40.828858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730307f-8006-427d-8786-88e4f34a624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:40.829458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730307f-8006-427d-8786-88e4f34a624c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:40.829734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730307f-8006-427d-8786-88e4f34a624c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:40.830033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730307f-8006-427d-8786-88e4f34a624c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:40.830755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730307f-8006-427d-8786-88e4f34a624c None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=367396f254c142b8ad02553e42e76250, user.id=5fcdf00e89ce4ea492ab5c4e20a65d76) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:40.831060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730307f-8006-427d-8786-88e4f34a624c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:40.832479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730307f-8006-427d-8786-88e4f34a624c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:40.837938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 574/1146] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:06:40 2022] GET /identity/v3/role_assignments?user.id=5fcdf00e89ce4ea492ab5c4e20a65d76&scope.project.id=367396f254c142b8ad02553e42e76250&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:42.742607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea16ee1-c308-4fd0-951d-6e96042df50f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.743253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea16ee1-c308-4fd0-951d-6e96042df50f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.743253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea16ee1-c308-4fd0-951d-6e96042df50f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.744459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 573/1147] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:42.750468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd450b5-e2f3-4924-bcf8-2e983568d7e8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:42.750891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd450b5-e2f3-4924-bcf8-2e983568d7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:42.751012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd450b5-e2f3-4924-bcf8-2e983568d7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:42.758221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:42.768874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fdd450b5-e2f3-4924-bcf8-2e983568d7e8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:42.774279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fdd450b5-e2f3-4924-bcf8-2e983568d7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:42.836845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 575/1148] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:06:43.108597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea16ee1-c308-4fd0-951d-6e96042df50f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.210760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea16ee1-c308-4fd0-951d-6e96042df50f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.244561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251898f-535a-475e-9ec2-c831b430a5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.245177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251898f-535a-475e-9ec2-c831b430a5eb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.245478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251898f-535a-475e-9ec2-c831b430a5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.245745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251898f-535a-475e-9ec2-c831b430a5eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.256222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251898f-535a-475e-9ec2-c831b430a5eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-0251898f-535a-475e-9ec2-c831b430a5eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:06:43.261047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:43.262303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:43.263496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:43.265088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 574/1149] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:06:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:43.269010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd450b5-e2f3-4924-bcf8-2e983568d7e8 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.368859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd450b5-e2f3-4924-bcf8-2e983568d7e8 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.402246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e56a8c-26ca-4547-a897-e57a668f0402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.402847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e56a8c-26ca-4547-a897-e57a668f0402 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.403124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e56a8c-26ca-4547-a897-e57a668f0402 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.403393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e56a8c-26ca-4547-a897-e57a668f0402 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.404057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e56a8c-26ca-4547-a897-e57a668f0402 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:43.404356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e56a8c-26ca-4547-a897-e57a668f0402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.406399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e56a8c-26ca-4547-a897-e57a668f0402 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.407849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:43.415464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 576/1150] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:06:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:43.421350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251898f-535a-475e-9ec2-c831b430a5eb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.469979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:43.521700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251898f-535a-475e-9ec2-c831b430a5eb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.557159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee99971-f30e-4921-8c64-7077a1f01411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.557782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee99971-f30e-4921-8c64-7077a1f01411 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.558176 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee99971-f30e-4921-8c64-7077a1f01411 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.558509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee99971-f30e-4921-8c64-7077a1f01411 None admin] PATH_INFO: `/v3/users/5fcdf00e89ce4ea492ab5c4e20a65d76` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.559780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee99971-f30e-4921-8c64-7077a1f01411 None admin] RBAC: Authorizing `identity:get_user(user_id=5fcdf00e89ce4ea492ab5c4e20a65d76)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.561925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee99971-f30e-4921-8c64-7077a1f01411 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.563734 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 575/1151] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:43 2022] GET /identity/v3/users/5fcdf00e89ce4ea492ab5c4e20a65d76 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:43.569534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e56a8c-26ca-4547-a897-e57a668f0402 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.676647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e56a8c-26ca-4547-a897-e57a668f0402 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.711232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18063b1d-52bc-43eb-ac72-87222642673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.711852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18063b1d-52bc-43eb-ac72-87222642673a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.712128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18063b1d-52bc-43eb-ac72-87222642673a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.712398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18063b1d-52bc-43eb-ac72-87222642673a None admin] PATH_INFO: `/v3/projects/367396f254c142b8ad02553e42e76250` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.713509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18063b1d-52bc-43eb-ac72-87222642673a None admin] RBAC: Authorizing `identity:get_project(project_id=367396f254c142b8ad02553e42e76250)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.714961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18063b1d-52bc-43eb-ac72-87222642673a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.716593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 577/1152] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:06:43 2022] GET /identity/v3/projects/367396f254c142b8ad02553e42e76250 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:43.722808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee99971-f30e-4921-8c64-7077a1f01411 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:43.822472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee99971-f30e-4921-8c64-7077a1f01411 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:43.837061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:43.857140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cfd50-fb15-4274-ae7e-2e4f2cbee814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:43.857824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cfd50-fb15-4274-ae7e-2e4f2cbee814 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:43.858169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cfd50-fb15-4274-ae7e-2e4f2cbee814 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:43.858445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cfd50-fb15-4274-ae7e-2e4f2cbee814 None admin] PATH_INFO: `/v3/projects/367396f254c142b8ad02553e42e76250/users/5fcdf00e89ce4ea492ab5c4e20a65d76/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:43.860384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cfd50-fb15-4274-ae7e-2e4f2cbee814 None admin] RBAC: Authorizing `identity:create_grant(project_id=367396f254c142b8ad02553e42e76250, user_id=5fcdf00e89ce4ea492ab5c4e20a65d76, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:43.861859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cfd50-fb15-4274-ae7e-2e4f2cbee814 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:43.952133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 576/1153] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:06:43 2022] PUT /identity/v3/projects/367396f254c142b8ad02553e42e76250/users/5fcdf00e89ce4ea492ab5c4e20a65d76/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:06:45.867870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaf758-eef7-4b6d-b79b-d6a8ddf6abf0 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:45.868431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaf758-eef7-4b6d-b79b-d6a8ddf6abf0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:45.868431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaf758-eef7-4b6d-b79b-d6a8ddf6abf0 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:45.869761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 578/1154] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:45.875700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59890753-189a-4076-a940-612c2b331e6c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:45.876139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59890753-189a-4076-a940-612c2b331e6c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:45.876245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59890753-189a-4076-a940-612c2b331e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:45.894674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-59890753-189a-4076-a940-612c2b331e6c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:45.895856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-59890753-189a-4076-a940-612c2b331e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:45.962431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 577/1155] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:46.238448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaf758-eef7-4b6d-b79b-d6a8ddf6abf0 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:46.307541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:46.348955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaf758-eef7-4b6d-b79b-d6a8ddf6abf0 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:46.390579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e6cbb-f0a4-4563-9cca-82588a9aec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:46.391213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e6cbb-f0a4-4563-9cca-82588a9aec32 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:46.391495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e6cbb-f0a4-4563-9cca-82588a9aec32 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:46.391768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e6cbb-f0a4-4563-9cca-82588a9aec32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:46.403292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e6cbb-f0a4-4563-9cca-82588a9aec32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-2d8e6cbb-f0a4-4563-9cca-82588a9aec32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:06:46.408840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:46.410078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:06:46.411163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 579/1156] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:06:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:46.416233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59890753-189a-4076-a940-612c2b331e6c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:46.523238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59890753-189a-4076-a940-612c2b331e6c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:46.525424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:46.561644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cdbb5-281a-4e6d-b509-dca42ab9820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:46.562277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cdbb5-281a-4e6d-b509-dca42ab9820a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:46.562558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cdbb5-281a-4e6d-b509-dca42ab9820a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:46.562830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cdbb5-281a-4e6d-b509-dca42ab9820a None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:46.563529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cdbb5-281a-4e6d-b509-dca42ab9820a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:46.563834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cdbb5-281a-4e6d-b509-dca42ab9820a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:46.565275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cdbb5-281a-4e6d-b509-dca42ab9820a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:46.574719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 578/1157] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:06:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:46.581450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e6cbb-f0a4-4563-9cca-82588a9aec32 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:46.687232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e6cbb-f0a4-4563-9cca-82588a9aec32 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:46.711363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:46.721901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10c005-5231-4afd-8e3b-109e8ac142f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:46.724594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10c005-5231-4afd-8e3b-109e8ac142f4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:46.724594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10c005-5231-4afd-8e3b-109e8ac142f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:46.724594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10c005-5231-4afd-8e3b-109e8ac142f4 None admin] PATH_INFO: `/v3/users/5fcdf00e89ce4ea492ab5c4e20a65d76` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:46.724594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10c005-5231-4afd-8e3b-109e8ac142f4 None admin] RBAC: Authorizing `identity:get_user(user_id=5fcdf00e89ce4ea492ab5c4e20a65d76)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:46.725564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10c005-5231-4afd-8e3b-109e8ac142f4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:46.727241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 580/1158] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:06:46 2022] GET /identity/v3/users/5fcdf00e89ce4ea492ab5c4e20a65d76 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:46.733287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cdbb5-281a-4e6d-b509-dca42ab9820a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:46.834381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cdbb5-281a-4e6d-b509-dca42ab9820a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:46.867022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b640b-7691-4219-a133-8c354a2da099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:46.867629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b640b-7691-4219-a133-8c354a2da099 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:46.867905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b640b-7691-4219-a133-8c354a2da099 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:46.868179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b640b-7691-4219-a133-8c354a2da099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:46.901777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:46.936687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b640b-7691-4219-a133-8c354a2da099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:46.938180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b640b-7691-4219-a133-8c354a2da099 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:47.035906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 579/1159] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:06:47.041888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10c005-5231-4afd-8e3b-109e8ac142f4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:47.143299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10c005-5231-4afd-8e3b-109e8ac142f4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:47.177791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901ff00-279a-4d84-b81f-74160a16866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:47.178454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901ff00-279a-4d84-b81f-74160a16866f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:47.178731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901ff00-279a-4d84-b81f-74160a16866f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:47.179002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901ff00-279a-4d84-b81f-74160a16866f None admin] PATH_INFO: `/v3/projects/367396f254c142b8ad02553e42e76250` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:47.180107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901ff00-279a-4d84-b81f-74160a16866f None admin] RBAC: Authorizing `identity:get_project(project_id=367396f254c142b8ad02553e42e76250)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:47.181829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901ff00-279a-4d84-b81f-74160a16866f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:47.183512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 581/1160] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:06:47 2022] GET /identity/v3/projects/367396f254c142b8ad02553e42e76250 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:47.190309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b640b-7691-4219-a133-8c354a2da099 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:47.255187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:47.290045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b640b-7691-4219-a133-8c354a2da099 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:47.324532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebe29e-2fe1-49cd-9af6-fd6cffa72dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:47.325125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebe29e-2fe1-49cd-9af6-fd6cffa72dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:47.325400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebe29e-2fe1-49cd-9af6-fd6cffa72dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:47.325667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebe29e-2fe1-49cd-9af6-fd6cffa72dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:47.326443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebe29e-2fe1-49cd-9af6-fd6cffa72dc5 None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=367396f254c142b8ad02553e42e76250, user.id=5fcdf00e89ce4ea492ab5c4e20a65d76) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:47.326747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebe29e-2fe1-49cd-9af6-fd6cffa72dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:47.328164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebe29e-2fe1-49cd-9af6-fd6cffa72dc5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:47.334143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 580/1161] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:06:47 2022] GET /identity/v3/role_assignments?user.id=5fcdf00e89ce4ea492ab5c4e20a65d76&scope.project.id=367396f254c142b8ad02553e42e76250&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:49.290401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fee9fc-56c3-461f-ac81-34a77ebb0460 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:49.290963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fee9fc-56c3-461f-ac81-34a77ebb0460 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:49.290963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fee9fc-56c3-461f-ac81-34a77ebb0460 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:49.292296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 582/1162] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:49.297930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713b566-3c06-4f8c-90be-5f5b767de8ba None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:49.298369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713b566-3c06-4f8c-90be-5f5b767de8ba None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:49.298464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713b566-3c06-4f8c-90be-5f5b767de8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:49.316937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d713b566-3c06-4f8c-90be-5f5b767de8ba None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:49.318186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d713b566-3c06-4f8c-90be-5f5b767de8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:49.389962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 581/1163] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:49.658634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fee9fc-56c3-461f-ac81-34a77ebb0460 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:49.723971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:49.768643 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fee9fc-56c3-461f-ac81-34a77ebb0460 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:49.802930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e148fdd-e6b8-4f88-9ad5-a85c5f824202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:49.803524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e148fdd-e6b8-4f88-9ad5-a85c5f824202 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:49.803806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e148fdd-e6b8-4f88-9ad5-a85c5f824202 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:49.804082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e148fdd-e6b8-4f88-9ad5-a85c5f824202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:49.871998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e148fdd-e6b8-4f88-9ad5-a85c5f824202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:49.874008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e148fdd-e6b8-4f88-9ad5-a85c5f824202 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:49.931129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:49.972749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 583/1164] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:06:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:06:49.978796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713b566-3c06-4f8c-90be-5f5b767de8ba None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:50.044958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:50.086778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713b566-3c06-4f8c-90be-5f5b767de8ba None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:50.121071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35b5b1-1180-474d-ab39-6b9e703ec653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:50.121688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35b5b1-1180-474d-ab39-6b9e703ec653 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:50.121996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35b5b1-1180-474d-ab39-6b9e703ec653 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:50.122268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35b5b1-1180-474d-ab39-6b9e703ec653 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:50.129502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35b5b1-1180-474d-ab39-6b9e703ec653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:50.130971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35b5b1-1180-474d-ab39-6b9e703ec653 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-be35b5b1-1180-474d-ab39-6b9e703ec653 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:06:50.142125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:06:50.143338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:06:50.144212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 582/1165] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:06:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:06:50.149296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e148fdd-e6b8-4f88-9ad5-a85c5f824202 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:50.248825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e148fdd-e6b8-4f88-9ad5-a85c5f824202 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:50.286016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4e629-917c-45d9-a57d-6dfec9e32d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:50.286682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4e629-917c-45d9-a57d-6dfec9e32d04 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:50.286962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4e629-917c-45d9-a57d-6dfec9e32d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:50.287232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4e629-917c-45d9-a57d-6dfec9e32d04 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:50.287890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4e629-917c-45d9-a57d-6dfec9e32d04 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:06:50.288195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4e629-917c-45d9-a57d-6dfec9e32d04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:50.289685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4e629-917c-45d9-a57d-6dfec9e32d04 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:50.304679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 584/1166] 10.222.0.83 () {56 vars in 1115 bytes} [Wed Aug 24 11:06:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:52.221695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3457cc-ca77-4b7d-8a4b-8e00eb4dd857 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:52.222320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3457cc-ca77-4b7d-8a4b-8e00eb4dd857 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:52.222320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3457cc-ca77-4b7d-8a4b-8e00eb4dd857 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:52.223618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 583/1167] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:52.228930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c6525-3e5b-4c3b-be3b-0c5d87456c00 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:52.229351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c6525-3e5b-4c3b-be3b-0c5d87456c00 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:52.229458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c6525-3e5b-4c3b-be3b-0c5d87456c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:52.248787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4e3c6525-3e5b-4c3b-be3b-0c5d87456c00 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:52.249991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4e3c6525-3e5b-4c3b-be3b-0c5d87456c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:52.267872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:52.323275 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 585/1168] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:52.600426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3457cc-ca77-4b7d-8a4b-8e00eb4dd857 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:52.702893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3457cc-ca77-4b7d-8a4b-8e00eb4dd857 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:52.723407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:52.738070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741a449-aeb2-41c5-b2c9-10534bccaf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:52.738681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741a449-aeb2-41c5-b2c9-10534bccaf32 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:52.738963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741a449-aeb2-41c5-b2c9-10534bccaf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:52.739236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741a449-aeb2-41c5-b2c9-10534bccaf32 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:52.739899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741a449-aeb2-41c5-b2c9-10534bccaf32 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:52.741386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741a449-aeb2-41c5-b2c9-10534bccaf32 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:52.814803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 584/1169] 10.222.0.83 () {60 vars in 1138 bytes} [Wed Aug 24 11:06:52 2022] POST /identity/v3/domains => generated 267 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:54.760049 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5e30-7105-4739-bed6-aa2d8aaac4cb None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:54.760553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5e30-7105-4739-bed6-aa2d8aaac4cb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:54.760553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5e30-7105-4739-bed6-aa2d8aaac4cb None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:54.761861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 586/1170] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:54.767817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93c8a1-385c-47ce-a81a-335c1bb75e29 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:54.768374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93c8a1-385c-47ce-a81a-335c1bb75e29 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:54.768374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93c8a1-385c-47ce-a81a-335c1bb75e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:54.786939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5b93c8a1-385c-47ce-a81a-335c1bb75e29 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:54.788131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5b93c8a1-385c-47ce-a81a-335c1bb75e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:54.855313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 585/1171] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:55.127244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb5e30-7105-4739-bed6-aa2d8aaac4cb None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:55.231182 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb5e30-7105-4739-bed6-aa2d8aaac4cb None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:55.268398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5b0a7-f2a8-4abc-8e39-9502ccf613c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:55.269024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5b0a7-f2a8-4abc-8e39-9502ccf613c8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:55.269302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5b0a7-f2a8-4abc-8e39-9502ccf613c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:55.269577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5b0a7-f2a8-4abc-8e39-9502ccf613c8 None admin] PATH_INFO: `/v3/domains/ab7fa96f3aa54f5eb863b43fe1356028` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:55.280048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5b0a7-f2a8-4abc-8e39-9502ccf613c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ab7fa96f3aa54f5eb863b43fe1356028)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:55.281539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5b0a7-f2a8-4abc-8e39-9502ccf613c8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:55.283428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 587/1172] 10.222.0.83 () {56 vars in 1183 bytes} [Wed Aug 24 11:06:55 2022] GET /identity/v3/domains/ab7fa96f3aa54f5eb863b43fe1356028 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:55.289131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93c8a1-385c-47ce-a81a-335c1bb75e29 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:55.388700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93c8a1-385c-47ce-a81a-335c1bb75e29 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:55.399888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:55.424281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3d33c-ff2f-457b-8ad9-832a3911694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:55.424896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3d33c-ff2f-457b-8ad9-832a3911694f None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:55.425184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3d33c-ff2f-457b-8ad9-832a3911694f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:55.425449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3d33c-ff2f-457b-8ad9-832a3911694f None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:55.426138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3d33c-ff2f-457b-8ad9-832a3911694f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:55.427594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3d33c-ff2f-457b-8ad9-832a3911694f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:55.535939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 586/1173] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:06:55 2022] POST /identity/v3/projects => generated 361 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:06:57.505890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1ef6c-f2c2-4555-b26d-5f8b9c250eae None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:57.506460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1ef6c-f2c2-4555-b26d-5f8b9c250eae None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:57.506460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1ef6c-f2c2-4555-b26d-5f8b9c250eae None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:57.507757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 588/1174] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:06:57.513092 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68690d7-37c3-4e9a-9ebd-3c293f7c4462 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:57.513538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68690d7-37c3-4e9a-9ebd-3c293f7c4462 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:57.513743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68690d7-37c3-4e9a-9ebd-3c293f7c4462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:57.531765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e68690d7-37c3-4e9a-9ebd-3c293f7c4462 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:06:57.532952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e68690d7-37c3-4e9a-9ebd-3c293f7c4462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:06:57.599218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 587/1175] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:06:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:06:57.884021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1ef6c-f2c2-4555-b26d-5f8b9c250eae None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:57.900150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:57.992297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1ef6c-f2c2-4555-b26d-5f8b9c250eae None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:58.025967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640476e-cde3-4c79-8c17-92820498b40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:58.026596 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640476e-cde3-4c79-8c17-92820498b40d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:58.026874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640476e-cde3-4c79-8c17-92820498b40d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:58.027150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640476e-cde3-4c79-8c17-92820498b40d None admin] PATH_INFO: `/v3/domains/ab7fa96f3aa54f5eb863b43fe1356028` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:58.028303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640476e-cde3-4c79-8c17-92820498b40d None admin] RBAC: Authorizing `identity:get_domain(domain_id=ab7fa96f3aa54f5eb863b43fe1356028)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:58.030306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640476e-cde3-4c79-8c17-92820498b40d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:58.031939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 589/1176] 10.222.0.83 () {56 vars in 1183 bytes} [Wed Aug 24 11:06:57 2022] GET /identity/v3/domains/ab7fa96f3aa54f5eb863b43fe1356028 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:06:58.037825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68690d7-37c3-4e9a-9ebd-3c293f7c4462 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:06:58.139509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:06:58.148241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68690d7-37c3-4e9a-9ebd-3c293f7c4462 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:06:58.186091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0f1ed-afcb-471e-9fd6-ac9e0edc8f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:06:58.186755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0f1ed-afcb-471e-9fd6-ac9e0edc8f13 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:06:58.187033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0f1ed-afcb-471e-9fd6-ac9e0edc8f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:06:58.187311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0f1ed-afcb-471e-9fd6-ac9e0edc8f13 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:06:58.187976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0f1ed-afcb-471e-9fd6-ac9e0edc8f13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:06:58.189421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0f1ed-afcb-471e-9fd6-ac9e0edc8f13 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:06:58.276785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 588/1177] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:06:58 2022] POST /identity/v3/users => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:07:00.268423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eb7b9-8db4-48ea-b8ba-dc09640f43d6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:00.268955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eb7b9-8db4-48ea-b8ba-dc09640f43d6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:00.268955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eb7b9-8db4-48ea-b8ba-dc09640f43d6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:00.270410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 590/1178] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:07:00.276198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623e2f9-f565-4c2a-b61b-fbc29d33e7d6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:00.276804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623e2f9-f565-4c2a-b61b-fbc29d33e7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:00.276804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623e2f9-f565-4c2a-b61b-fbc29d33e7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:00.294935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e623e2f9-f565-4c2a-b61b-fbc29d33e7d6 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:00.296161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e623e2f9-f565-4c2a-b61b-fbc29d33e7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:00.363214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 589/1179] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:07:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:07:00.640139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eb7b9-8db4-48ea-b8ba-dc09640f43d6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:00.725138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:00.744658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eb7b9-8db4-48ea-b8ba-dc09640f43d6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:00.779919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a667f30-fb8f-4528-86da-2fff49589a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:00.780538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a667f30-fb8f-4528-86da-2fff49589a04 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:00.780817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a667f30-fb8f-4528-86da-2fff49589a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:00.781085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a667f30-fb8f-4528-86da-2fff49589a04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:00.791572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a667f30-fb8f-4528-86da-2fff49589a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8a667f30-fb8f-4528-86da-2fff49589a04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:07:00.796929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:07:00.798061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:07:00.799113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 591/1180] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:07:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:07:00.804226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623e2f9-f565-4c2a-b61b-fbc29d33e7d6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:00.883140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:00.905692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623e2f9-f565-4c2a-b61b-fbc29d33e7d6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:00.941821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecaf18-b694-4ac2-afa9-6f039ddb11c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:00.942449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecaf18-b694-4ac2-afa9-6f039ddb11c2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:00.942723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecaf18-b694-4ac2-afa9-6f039ddb11c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:00.942989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecaf18-b694-4ac2-afa9-6f039ddb11c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:00.943741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecaf18-b694-4ac2-afa9-6f039ddb11c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:00.944049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecaf18-b694-4ac2-afa9-6f039ddb11c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:00.945420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecaf18-b694-4ac2-afa9-6f039ddb11c2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:00.953652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 590/1181] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:07:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:00.959581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a667f30-fb8f-4528-86da-2fff49589a04 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:01.060752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a667f30-fb8f-4528-86da-2fff49589a04 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:01.096713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3210b94-9fd0-4190-959a-4a36c27dc369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:01.097339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3210b94-9fd0-4190-959a-4a36c27dc369 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.097616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3210b94-9fd0-4190-959a-4a36c27dc369 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.097976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3210b94-9fd0-4190-959a-4a36c27dc369 None admin] PATH_INFO: `/v3/users/3a3251e6c13d428b9174363a87b48e88` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.109980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:01.112343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3210b94-9fd0-4190-959a-4a36c27dc369 None admin] RBAC: Authorizing `identity:get_user(user_id=3a3251e6c13d428b9174363a87b48e88)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:01.113791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3210b94-9fd0-4190-959a-4a36c27dc369 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:01.115512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 592/1182] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:07:00 2022] GET /identity/v3/users/3a3251e6c13d428b9174363a87b48e88 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:01.121805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecaf18-b694-4ac2-afa9-6f039ddb11c2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:01.225746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecaf18-b694-4ac2-afa9-6f039ddb11c2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:01.259008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ebd59-5a65-4955-82ab-b944798b1a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:01.259612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ebd59-5a65-4955-82ab-b944798b1a95 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.259893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ebd59-5a65-4955-82ab-b944798b1a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.260163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ebd59-5a65-4955-82ab-b944798b1a95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.271534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:01.330380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ebd59-5a65-4955-82ab-b944798b1a95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:01.331840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ebd59-5a65-4955-82ab-b944798b1a95 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:01.430571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 591/1183] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:07:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:07:01.436644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3210b94-9fd0-4190-959a-4a36c27dc369 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:01.540865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3210b94-9fd0-4190-959a-4a36c27dc369 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:01.575084 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84f8ac-591c-4d29-9912-4afd936df62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:01.575708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84f8ac-591c-4d29-9912-4afd936df62e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.576020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84f8ac-591c-4d29-9912-4afd936df62e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.576827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84f8ac-591c-4d29-9912-4afd936df62e None admin] PATH_INFO: `/v3/projects/ff84559d35df492d817d6f4d1d6dc563` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.578030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84f8ac-591c-4d29-9912-4afd936df62e None admin] RBAC: Authorizing `identity:get_project(project_id=ff84559d35df492d817d6f4d1d6dc563)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:01.579466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84f8ac-591c-4d29-9912-4afd936df62e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:01.581120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 593/1184] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:07:01 2022] GET /identity/v3/projects/ff84559d35df492d817d6f4d1d6dc563 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:01.587567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ebd59-5a65-4955-82ab-b944798b1a95 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:01.629700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:01.688234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ebd59-5a65-4955-82ab-b944798b1a95 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:01.723196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08d1b8-f1be-40fa-a925-5d908a934308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:01.723801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08d1b8-f1be-40fa-a925-5d908a934308 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:01.724086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08d1b8-f1be-40fa-a925-5d908a934308 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:01.724359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08d1b8-f1be-40fa-a925-5d908a934308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:01.725101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08d1b8-f1be-40fa-a925-5d908a934308 None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=ff84559d35df492d817d6f4d1d6dc563, user.id=3a3251e6c13d428b9174363a87b48e88) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:01.725410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08d1b8-f1be-40fa-a925-5d908a934308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:01.726896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08d1b8-f1be-40fa-a925-5d908a934308 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:01.732499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 592/1185] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:07:01 2022] GET /identity/v3/role_assignments?user.id=3a3251e6c13d428b9174363a87b48e88&scope.project.id=ff84559d35df492d817d6f4d1d6dc563&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:03.770828 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf289e9-a193-4cca-bcb2-cacd2228bf92 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:03.771347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf289e9-a193-4cca-bcb2-cacd2228bf92 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:03.771347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf289e9-a193-4cca-bcb2-cacd2228bf92 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:03.772666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 594/1186] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:07:03.780818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ca612-b162-40ed-97d3-08225e2c5bba None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:03.781296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ca612-b162-40ed-97d3-08225e2c5bba None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:03.781419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ca612-b162-40ed-97d3-08225e2c5bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:03.802255 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-351ca612-b162-40ed-97d3-08225e2c5bba None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:03.803484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-351ca612-b162-40ed-97d3-08225e2c5bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:03.880043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 593/1187] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:07:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:07:04.148810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf289e9-a193-4cca-bcb2-cacd2228bf92 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:04.218116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:04.254330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf289e9-a193-4cca-bcb2-cacd2228bf92 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:04.288588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e1a0e-c48e-4611-8da1-371e22422979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:04.289218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e1a0e-c48e-4611-8da1-371e22422979 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:04.289499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e1a0e-c48e-4611-8da1-371e22422979 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:04.289829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e1a0e-c48e-4611-8da1-371e22422979 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:04.300584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e1a0e-c48e-4611-8da1-371e22422979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4b0e1a0e-c48e-4611-8da1-371e22422979 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:07:04.305560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:07:04.306698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:07:04.307734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 595/1188] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:07:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:07:04.313324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ca612-b162-40ed-97d3-08225e2c5bba None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:04.348277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:04.416897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ca612-b162-40ed-97d3-08225e2c5bba None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:04.452850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c3205-270d-4cdb-943a-78612c5e375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:04.453458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c3205-270d-4cdb-943a-78612c5e375f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:04.453738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c3205-270d-4cdb-943a-78612c5e375f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:04.454050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c3205-270d-4cdb-943a-78612c5e375f None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:04.454720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c3205-270d-4cdb-943a-78612c5e375f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:04.455029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c3205-270d-4cdb-943a-78612c5e375f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:04.456407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c3205-270d-4cdb-943a-78612c5e375f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:04.464932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 594/1189] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:07:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:04.470880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e1a0e-c48e-4611-8da1-371e22422979 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:04.581738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e1a0e-c48e-4611-8da1-371e22422979 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:04.610148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:04.618172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d2a35-cca0-4d32-a3f1-0606246dd848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:04.624171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d2a35-cca0-4d32-a3f1-0606246dd848 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:04.624171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d2a35-cca0-4d32-a3f1-0606246dd848 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:04.624171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d2a35-cca0-4d32-a3f1-0606246dd848 None admin] PATH_INFO: `/v3/users/3a3251e6c13d428b9174363a87b48e88` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:04.624171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d2a35-cca0-4d32-a3f1-0606246dd848 None admin] RBAC: Authorizing `identity:get_user(user_id=3a3251e6c13d428b9174363a87b48e88)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:04.624171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d2a35-cca0-4d32-a3f1-0606246dd848 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:04.624171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 596/1190] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:07:04 2022] GET /identity/v3/users/3a3251e6c13d428b9174363a87b48e88 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:04.630716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c3205-270d-4cdb-943a-78612c5e375f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:04.738430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:04.744539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c3205-270d-4cdb-943a-78612c5e375f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:04.779597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e9783-8b4e-41b1-a324-e3c91d102e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:04.780219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e9783-8b4e-41b1-a324-e3c91d102e41 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:04.780497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e9783-8b4e-41b1-a324-e3c91d102e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:04.780768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e9783-8b4e-41b1-a324-e3c91d102e41 None admin] PATH_INFO: `/v3/projects/ff84559d35df492d817d6f4d1d6dc563` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:04.781895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e9783-8b4e-41b1-a324-e3c91d102e41 None admin] RBAC: Authorizing `identity:get_project(project_id=ff84559d35df492d817d6f4d1d6dc563)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:04.783336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e9783-8b4e-41b1-a324-e3c91d102e41 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:04.784972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 595/1191] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:07:04 2022] GET /identity/v3/projects/ff84559d35df492d817d6f4d1d6dc563 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:04.791636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d2a35-cca0-4d32-a3f1-0606246dd848 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:04.892657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d2a35-cca0-4d32-a3f1-0606246dd848 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:04.927714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408be02e-2018-492e-93c6-45216c4e5c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:04.928380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408be02e-2018-492e-93c6-45216c4e5c5b None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:04.928659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408be02e-2018-492e-93c6-45216c4e5c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:04.929010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408be02e-2018-492e-93c6-45216c4e5c5b None admin] PATH_INFO: `/v3/projects/ff84559d35df492d817d6f4d1d6dc563/users/3a3251e6c13d428b9174363a87b48e88/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:04.931281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408be02e-2018-492e-93c6-45216c4e5c5b None admin] RBAC: Authorizing `identity:create_grant(project_id=ff84559d35df492d817d6f4d1d6dc563, user_id=3a3251e6c13d428b9174363a87b48e88, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:04.932775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408be02e-2018-492e-93c6-45216c4e5c5b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:05.031545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 597/1192] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:07:04 2022] PUT /identity/v3/projects/ff84559d35df492d817d6f4d1d6dc563/users/3a3251e6c13d428b9174363a87b48e88/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:07:06.942036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1851b71-7fa3-4aeb-b3c0-0123fa296a88 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:06.944055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1851b71-7fa3-4aeb-b3c0-0123fa296a88 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:06.944055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1851b71-7fa3-4aeb-b3c0-0123fa296a88 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:06.944514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 596/1193] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:07:06.949890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ea9db-2149-453c-b3ac-4bf077acdb87 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:06.950319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ea9db-2149-453c-b3ac-4bf077acdb87 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:06.950443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ea9db-2149-453c-b3ac-4bf077acdb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:06.970822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e45ea9db-2149-453c-b3ac-4bf077acdb87 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:07:06.972033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e45ea9db-2149-453c-b3ac-4bf077acdb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:07:07.023055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:07.039976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 598/1194] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:07:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:07:07.308627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1851b71-7fa3-4aeb-b3c0-0123fa296a88 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:07.412916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1851b71-7fa3-4aeb-b3c0-0123fa296a88 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:07.446789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05105748-335b-4f6a-95fc-a9b0a9413546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:07.447432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05105748-335b-4f6a-95fc-a9b0a9413546 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:07.447710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05105748-335b-4f6a-95fc-a9b0a9413546 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:07.447981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05105748-335b-4f6a-95fc-a9b0a9413546 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:07.459070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05105748-335b-4f6a-95fc-a9b0a9413546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-05105748-335b-4f6a-95fc-a9b0a9413546 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:07:07.463824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:07:07.464979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:07:07.466178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 597/1195] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:07:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:07:07.471690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ea9db-2149-453c-b3ac-4bf077acdb87 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:07.579754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ea9db-2149-453c-b3ac-4bf077acdb87 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:07.614202 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e7f2a-43a7-4056-8039-dd0ac37c9f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:07.614803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e7f2a-43a7-4056-8039-dd0ac37c9f2e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:07.615074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e7f2a-43a7-4056-8039-dd0ac37c9f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:07.615342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e7f2a-43a7-4056-8039-dd0ac37c9f2e None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:07.616005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e7f2a-43a7-4056-8039-dd0ac37c9f2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:07.616299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e7f2a-43a7-4056-8039-dd0ac37c9f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:07.617653 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e7f2a-43a7-4056-8039-dd0ac37c9f2e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:07.626229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 599/1196] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:07:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:07.632193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05105748-335b-4f6a-95fc-a9b0a9413546 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:07.659074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:07.735685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05105748-335b-4f6a-95fc-a9b0a9413546 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:07.771891 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8293e7-5063-4b2e-83da-7f98a5723cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:07.772544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8293e7-5063-4b2e-83da-7f98a5723cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:07.772827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8293e7-5063-4b2e-83da-7f98a5723cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:07.773104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8293e7-5063-4b2e-83da-7f98a5723cd4 None admin] PATH_INFO: `/v3/users/3a3251e6c13d428b9174363a87b48e88` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:07.774290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8293e7-5063-4b2e-83da-7f98a5723cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=3a3251e6c13d428b9174363a87b48e88)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:07.775740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8293e7-5063-4b2e-83da-7f98a5723cd4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:07.777331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 598/1197] 10.222.0.83 () {56 vars in 1177 bytes} [Wed Aug 24 11:07:07 2022] GET /identity/v3/users/3a3251e6c13d428b9174363a87b48e88 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:07.783313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e7f2a-43a7-4056-8039-dd0ac37c9f2e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:07.834153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:07.887314 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e7f2a-43a7-4056-8039-dd0ac37c9f2e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:07.921860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad298-2fa7-46d8-b765-ad1ae787cb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:07.922482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad298-2fa7-46d8-b765-ad1ae787cb3a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:07.922764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad298-2fa7-46d8-b765-ad1ae787cb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:07.923042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad298-2fa7-46d8-b765-ad1ae787cb3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:07.991589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad298-2fa7-46d8-b765-ad1ae787cb3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:07.993041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad298-2fa7-46d8-b765-ad1ae787cb3a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:08.041925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:08.095675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 600/1198] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:07:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:07:08.102058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8293e7-5063-4b2e-83da-7f98a5723cd4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:08.199209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:08.208506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8293e7-5063-4b2e-83da-7f98a5723cd4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:08.242742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371e24f-9378-4879-8e37-6a5813eb2c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:08.243378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371e24f-9378-4879-8e37-6a5813eb2c5b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:08.243661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371e24f-9378-4879-8e37-6a5813eb2c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:08.243936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371e24f-9378-4879-8e37-6a5813eb2c5b None admin] PATH_INFO: `/v3/projects/ff84559d35df492d817d6f4d1d6dc563` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:08.245058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371e24f-9378-4879-8e37-6a5813eb2c5b None admin] RBAC: Authorizing `identity:get_project(project_id=ff84559d35df492d817d6f4d1d6dc563)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:08.246566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371e24f-9378-4879-8e37-6a5813eb2c5b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:08.248829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 599/1199] 10.222.0.83 () {56 vars in 1186 bytes} [Wed Aug 24 11:07:08 2022] GET /identity/v3/projects/ff84559d35df492d817d6f4d1d6dc563 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:08.255087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad298-2fa7-46d8-b765-ad1ae787cb3a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:07:08.361293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad298-2fa7-46d8-b765-ad1ae787cb3a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:07:08.395920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cbec0-3b18-46b3-adbe-dfc1e0154a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:07:08.396534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cbec0-3b18-46b3-adbe-dfc1e0154a13 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:07:08.396816 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cbec0-3b18-46b3-adbe-dfc1e0154a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:07:08.397117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cbec0-3b18-46b3-adbe-dfc1e0154a13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:07:08.397853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683cbec0-3b18-46b3-adbe-dfc1e0154a13 None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=ff84559d35df492d817d6f4d1d6dc563, user.id=3a3251e6c13d428b9174363a87b48e88) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:07:08.398191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683cbec0-3b18-46b3-adbe-dfc1e0154a13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:07:08.399657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683cbec0-3b18-46b3-adbe-dfc1e0154a13 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:07:08.406753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 601/1200] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:07:08 2022] GET /identity/v3/role_assignments?user.id=3a3251e6c13d428b9174363a87b48e88&scope.project.id=ff84559d35df492d817d6f4d1d6dc563&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:07:18.238391 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=20 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:07:18.405010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:24.808084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac05b8b-bc81-4b6f-bfde-5089ec8af77e None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.815497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac05b8b-bc81-4b6f-bfde-5089ec8af77e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.815497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac05b8b-bc81-4b6f-bfde-5089ec8af77e None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.815497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 600/1201] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:24.815829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721c42-d865-4406-bac2-f4f06a23dc32 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:24.815829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721c42-d865-4406-bac2-f4f06a23dc32 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:24.816298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721c42-d865-4406-bac2-f4f06a23dc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:24.834734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-29721c42-d865-4406-bac2-f4f06a23dc32 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:24.835956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-29721c42-d865-4406-bac2-f4f06a23dc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:24.906875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 602/1202] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:08:25.176109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac05b8b-bc81-4b6f-bfde-5089ec8af77e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.283366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac05b8b-bc81-4b6f-bfde-5089ec8af77e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.323189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f079d-611f-43d8-8513-6b2b7dcda459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.323828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f079d-611f-43d8-8513-6b2b7dcda459 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.324107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f079d-611f-43d8-8513-6b2b7dcda459 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.324380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f079d-611f-43d8-8513-6b2b7dcda459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.335907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f079d-611f-43d8-8513-6b2b7dcda459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.337448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f079d-611f-43d8-8513-6b2b7dcda459 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-680f079d-611f-43d8-8513-6b2b7dcda459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:25.352194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:25.353468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:25.354594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:25.354594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:25.354594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:25.354594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 601/1203] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:08:25.359483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721c42-d865-4406-bac2-f4f06a23dc32 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.463221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721c42-d865-4406-bac2-f4f06a23dc32 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.494494 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:25.504841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce6b1b-4b19-4773-bc08-8f0008f85696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.505911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce6b1b-4b19-4773-bc08-8f0008f85696 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.506429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce6b1b-4b19-4773-bc08-8f0008f85696 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.506974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce6b1b-4b19-4773-bc08-8f0008f85696 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.508133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce6b1b-4b19-4773-bc08-8f0008f85696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:25.508689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce6b1b-4b19-4773-bc08-8f0008f85696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.510824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce6b1b-4b19-4773-bc08-8f0008f85696 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.523614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 603/1204] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:25.529598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f079d-611f-43d8-8513-6b2b7dcda459 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:25.586530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:25.633389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f079d-611f-43d8-8513-6b2b7dcda459 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:25.667352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a008be3-da81-4044-99ce-76dccd0eac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:25.667956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a008be3-da81-4044-99ce-76dccd0eac27 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:25.668226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a008be3-da81-4044-99ce-76dccd0eac27 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:25.668508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a008be3-da81-4044-99ce-76dccd0eac27 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:25.669128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a008be3-da81-4044-99ce-76dccd0eac27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:25.670618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a008be3-da81-4044-99ce-76dccd0eac27 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:25.807632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 602/1205] 10.222.0.83 () {60 vars in 1132 bytes} [Wed Aug 24 11:08:25 2022] POST /identity/v3/users => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:27.762373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ddaf5-3a3f-4efe-a407-fbd50828bf07 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:27.762898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ddaf5-3a3f-4efe-a407-fbd50828bf07 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:27.762898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ddaf5-3a3f-4efe-a407-fbd50828bf07 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:27.764203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 604/1206] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:27.770412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef065a-bc4f-4900-ba89-33eb852e5dd2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:27.770819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef065a-bc4f-4900-ba89-33eb852e5dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:27.770957 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef065a-bc4f-4900-ba89-33eb852e5dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:27.789301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-67ef065a-bc4f-4900-ba89-33eb852e5dd2 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:27.790605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-67ef065a-bc4f-4900-ba89-33eb852e5dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:27.867447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 603/1207] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:08:28.140144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ddaf5-3a3f-4efe-a407-fbd50828bf07 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:28.252737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ddaf5-3a3f-4efe-a407-fbd50828bf07 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:28.287894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8525f0-ca36-426b-8d73-edbced536b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:28.288553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8525f0-ca36-426b-8d73-edbced536b9e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:28.288832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8525f0-ca36-426b-8d73-edbced536b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:28.289106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8525f0-ca36-426b-8d73-edbced536b9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:28.300569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8525f0-ca36-426b-8d73-edbced536b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-dd8525f0-ca36-426b-8d73-edbced536b9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:08:28.305625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:28.310006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:28.311069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 605/1208] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:08:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:28.313028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef065a-bc4f-4900-ba89-33eb852e5dd2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:28.357757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:28.419873 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef065a-bc4f-4900-ba89-33eb852e5dd2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:28.455560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d220b-d8ad-43bd-906f-2e7225b2ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:28.456174 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d220b-d8ad-43bd-906f-2e7225b2ea60 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:28.456455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d220b-d8ad-43bd-906f-2e7225b2ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:28.456726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d220b-d8ad-43bd-906f-2e7225b2ea60 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:28.457432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d220b-d8ad-43bd-906f-2e7225b2ea60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:28.457989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d220b-d8ad-43bd-906f-2e7225b2ea60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:28.459909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d220b-d8ad-43bd-906f-2e7225b2ea60 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:28.471233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 604/1209] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:08:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:28.477109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8525f0-ca36-426b-8d73-edbced536b9e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:28.529691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:28.582031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8525f0-ca36-426b-8d73-edbced536b9e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:28.617843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60538f61-62e1-489b-9a0f-bd4443dee780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:28.618519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60538f61-62e1-489b-9a0f-bd4443dee780 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:28.618795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60538f61-62e1-489b-9a0f-bd4443dee780 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:28.619066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60538f61-62e1-489b-9a0f-bd4443dee780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:28.629782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60538f61-62e1-489b-9a0f-bd4443dee780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:28.631268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60538f61-62e1-489b-9a0f-bd4443dee780 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-60538f61-62e1-489b-9a0f-bd4443dee780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:28.645214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:28.646384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:28.647404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:28.647404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:28.647404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:28.647404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 606/1210] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:28.652359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d220b-d8ad-43bd-906f-2e7225b2ea60 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:28.756945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:28.760132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d220b-d8ad-43bd-906f-2e7225b2ea60 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:28.794986 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166b8b6-2736-4d13-be2e-76f08c0d60af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:28.795623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166b8b6-2736-4d13-be2e-76f08c0d60af None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:28.795840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166b8b6-2736-4d13-be2e-76f08c0d60af None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:28.796115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166b8b6-2736-4d13-be2e-76f08c0d60af None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:28.796764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166b8b6-2736-4d13-be2e-76f08c0d60af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:28.797068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166b8b6-2736-4d13-be2e-76f08c0d60af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:28.799084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166b8b6-2736-4d13-be2e-76f08c0d60af None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:28.812240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 605/1211] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:28.818165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60538f61-62e1-489b-9a0f-bd4443dee780 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:28.922437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60538f61-62e1-489b-9a0f-bd4443dee780 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:28.924294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:28.957965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f596214-ba8d-4205-bc25-ace52a6453a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:28.958590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f596214-ba8d-4205-bc25-ace52a6453a8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:28.958869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f596214-ba8d-4205-bc25-ace52a6453a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:28.959140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f596214-ba8d-4205-bc25-ace52a6453a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:28.966949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f596214-ba8d-4205-bc25-ace52a6453a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:28.968322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f596214-ba8d-4205-bc25-ace52a6453a8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-6f596214-ba8d-4205-bc25-ace52a6453a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:08:28.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:28.981317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:28.982395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 607/1212] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:28.987219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166b8b6-2736-4d13-be2e-76f08c0d60af None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.097821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166b8b6-2736-4d13-be2e-76f08c0d60af None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.132021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703f41a-69a5-4c06-8aa6-2588b4c0160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.132629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703f41a-69a5-4c06-8aa6-2588b4c0160c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.132909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703f41a-69a5-4c06-8aa6-2588b4c0160c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.133182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703f41a-69a5-4c06-8aa6-2588b4c0160c None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.133949 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703f41a-69a5-4c06-8aa6-2588b4c0160c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:29.134258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703f41a-69a5-4c06-8aa6-2588b4c0160c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.135694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703f41a-69a5-4c06-8aa6-2588b4c0160c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.154475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 606/1213] 10.222.0.83 () {56 vars in 1143 bytes} [Wed Aug 24 11:08:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:29.161166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f596214-ba8d-4205-bc25-ace52a6453a8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.265151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f596214-ba8d-4205-bc25-ace52a6453a8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.300716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6843d-c077-46f1-b22b-548812586884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.301323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6843d-c077-46f1-b22b-548812586884 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.301605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6843d-c077-46f1-b22b-548812586884 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.301894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6843d-c077-46f1-b22b-548812586884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.334162 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:29.381361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6843d-c077-46f1-b22b-548812586884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.383825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6843d-c077-46f1-b22b-548812586884 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.495780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 608/1214] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:08:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:08:29.502416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703f41a-69a5-4c06-8aa6-2588b4c0160c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.523252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:29.616018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703f41a-69a5-4c06-8aa6-2588b4c0160c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.657452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316b288-7e58-46da-8238-a8633c3be3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.657452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316b288-7e58-46da-8238-a8633c3be3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.657452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316b288-7e58-46da-8238-a8633c3be3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.657452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316b288-7e58-46da-8238-a8633c3be3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.662053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316b288-7e58-46da-8238-a8633c3be3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.663507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316b288-7e58-46da-8238-a8633c3be3e3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-2316b288-7e58-46da-8238-a8633c3be3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:29.678495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:29.680543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:29.682218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:29.682218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:29.682218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:29.682218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 607/1215] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:08:29.685967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6843d-c077-46f1-b22b-548812586884 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.753638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:29.792429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6843d-c077-46f1-b22b-548812586884 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.830265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e509d-c133-4da5-b603-6b98e138f07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.831147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e509d-c133-4da5-b603-6b98e138f07c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.831147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e509d-c133-4da5-b603-6b98e138f07c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.831375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e509d-c133-4da5-b603-6b98e138f07c None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.832015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e509d-c133-4da5-b603-6b98e138f07c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:29.832315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e509d-c133-4da5-b603-6b98e138f07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:29.833669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e509d-c133-4da5-b603-6b98e138f07c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:29.848785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 609/1216] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:08:29.857950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316b288-7e58-46da-8238-a8633c3be3e3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:29.924640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:29.956552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316b288-7e58-46da-8238-a8633c3be3e3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:29.990803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732cacf-7da4-4cf6-b719-a0e73c98a714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:29.991429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732cacf-7da4-4cf6-b719-a0e73c98a714 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:29.991707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732cacf-7da4-4cf6-b719-a0e73c98a714 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:29.991977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732cacf-7da4-4cf6-b719-a0e73c98a714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:29.998858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732cacf-7da4-4cf6-b719-a0e73c98a714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:30.000322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732cacf-7da4-4cf6-b719-a0e73c98a714 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-c732cacf-7da4-4cf6-b719-a0e73c98a714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:08:30.013046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:30.019862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:30.020770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:08:30.020770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:08:30.020770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:08:30.020770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:08:30.020770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:08:30.020770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:08:30.020770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:30.020770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:30.020770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 608/1217] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:30.021288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e509d-c133-4da5-b603-6b98e138f07c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:30.132893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e509d-c133-4da5-b603-6b98e138f07c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:30.158605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:30.170301 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d8ef5-d3ac-469c-96a2-f1ecaed10045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:30.170909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d8ef5-d3ac-469c-96a2-f1ecaed10045 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:30.171190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d8ef5-d3ac-469c-96a2-f1ecaed10045 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:30.171502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d8ef5-d3ac-469c-96a2-f1ecaed10045 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:30.174392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d8ef5-d3ac-469c-96a2-f1ecaed10045 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:30.174392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d8ef5-d3ac-469c-96a2-f1ecaed10045 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:30.174611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d8ef5-d3ac-469c-96a2-f1ecaed10045 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:30.187327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 610/1218] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:08:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:30.193599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732cacf-7da4-4cf6-b719-a0e73c98a714 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:30.298743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732cacf-7da4-4cf6-b719-a0e73c98a714 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:30.329825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:30.336319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4358f4-3a87-425a-9fb5-e7767d544d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:30.337088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4358f4-3a87-425a-9fb5-e7767d544d42 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:30.337352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4358f4-3a87-425a-9fb5-e7767d544d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:30.337624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4358f4-3a87-425a-9fb5-e7767d544d42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:30.338400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4358f4-3a87-425a-9fb5-e7767d544d42 None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=eb76ad667a744126ab00b38c346e9017) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:30.338711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4358f4-3a87-425a-9fb5-e7767d544d42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:30.340202 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4358f4-3a87-425a-9fb5-e7767d544d42 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:30.346437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 609/1219] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:08:30 2022] GET /identity/v3/role_assignments?user.id=eb76ad667a744126ab00b38c346e9017&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:32.269369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970b50a-a178-40b2-a4ff-349e18607ac3 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:32.270006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970b50a-a178-40b2-a4ff-349e18607ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:32.270006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970b50a-a178-40b2-a4ff-349e18607ac3 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:32.271293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 611/1220] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:32.276659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a083afd-bf91-4deb-b20e-bc5c65e06bae None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:32.277083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a083afd-bf91-4deb-b20e-bc5c65e06bae None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:32.277234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a083afd-bf91-4deb-b20e-bc5c65e06bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:32.296784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7a083afd-bf91-4deb-b20e-bc5c65e06bae None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:32.304779 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7a083afd-bf91-4deb-b20e-bc5c65e06bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:32.370707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 610/1221] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:08:32.644222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970b50a-a178-40b2-a4ff-349e18607ac3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:32.751420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970b50a-a178-40b2-a4ff-349e18607ac3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:32.787669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929dd37d-5c35-4209-a127-648ae8ad5964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:32.788358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dd37d-5c35-4209-a127-648ae8ad5964 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:32.788636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dd37d-5c35-4209-a127-648ae8ad5964 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:32.793327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dd37d-5c35-4209-a127-648ae8ad5964 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:32.801055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929dd37d-5c35-4209-a127-648ae8ad5964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-929dd37d-5c35-4209-a127-648ae8ad5964 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:08:32.806407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:32.807612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:32.808733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 612/1222] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:08:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:32.817197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a083afd-bf91-4deb-b20e-bc5c65e06bae None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:32.918884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a083afd-bf91-4deb-b20e-bc5c65e06bae None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:32.944724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:32.957256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dd131-a1fe-4879-adfe-296fd752328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:32.957894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dd131-a1fe-4879-adfe-296fd752328f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:32.958190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dd131-a1fe-4879-adfe-296fd752328f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:32.958463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dd131-a1fe-4879-adfe-296fd752328f None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:32.959147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427dd131-a1fe-4879-adfe-296fd752328f None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:32.959453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427dd131-a1fe-4879-adfe-296fd752328f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:32.960966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427dd131-a1fe-4879-adfe-296fd752328f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:32.970471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 611/1223] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:08:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:32.976768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929dd37d-5c35-4209-a127-648ae8ad5964 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:33.025154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:33.088702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929dd37d-5c35-4209-a127-648ae8ad5964 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:33.123099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c3f5-77d8-4781-adbe-275998a8662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:33.123722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c3f5-77d8-4781-adbe-275998a8662c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:33.123998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c3f5-77d8-4781-adbe-275998a8662c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:33.124265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c3f5-77d8-4781-adbe-275998a8662c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:33.134984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9c3f5-77d8-4781-adbe-275998a8662c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:33.136449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9c3f5-77d8-4781-adbe-275998a8662c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-fac9c3f5-77d8-4781-adbe-275998a8662c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:33.152463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:33.153776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:33.155090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:33.155090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:33.155090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:33.155090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 613/1224] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:33.159733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dd131-a1fe-4879-adfe-296fd752328f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:33.263350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dd131-a1fe-4879-adfe-296fd752328f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:33.298038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedd9c1-4b3b-45e3-b69f-6cadc88ddb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:33.298953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedd9c1-4b3b-45e3-b69f-6cadc88ddb74 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:33.298953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedd9c1-4b3b-45e3-b69f-6cadc88ddb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:33.299185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedd9c1-4b3b-45e3-b69f-6cadc88ddb74 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:33.299865 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedd9c1-4b3b-45e3-b69f-6cadc88ddb74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:33.300168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedd9c1-4b3b-45e3-b69f-6cadc88ddb74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:33.301606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedd9c1-4b3b-45e3-b69f-6cadc88ddb74 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:33.314710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 612/1225] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:33.320617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c3f5-77d8-4781-adbe-275998a8662c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:33.418560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:33.426266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c3f5-77d8-4781-adbe-275998a8662c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:33.461287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d7065-d4b8-4b5c-a0b4-e174a609c671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:33.472474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d7065-d4b8-4b5c-a0b4-e174a609c671 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:33.472474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d7065-d4b8-4b5c-a0b4-e174a609c671 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:33.472474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d7065-d4b8-4b5c-a0b4-e174a609c671 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:33.472474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d7065-d4b8-4b5c-a0b4-e174a609c671 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:33.472474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d7065-d4b8-4b5c-a0b4-e174a609c671 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-696d7065-d4b8-4b5c-a0b4-e174a609c671 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:08:33.482568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:33.483732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:33.485396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 614/1226] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:33.490239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedd9c1-4b3b-45e3-b69f-6cadc88ddb74 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:33.547047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:33.607399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedd9c1-4b3b-45e3-b69f-6cadc88ddb74 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:33.650456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c76af-9f7b-4103-89ed-d314c23147e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:33.651055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c76af-9f7b-4103-89ed-d314c23147e0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:33.651329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c76af-9f7b-4103-89ed-d314c23147e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:33.651639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c76af-9f7b-4103-89ed-d314c23147e0 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:33.652371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c76af-9f7b-4103-89ed-d314c23147e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:33.652671 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c76af-9f7b-4103-89ed-d314c23147e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:33.654127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c76af-9f7b-4103-89ed-d314c23147e0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:33.672727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 613/1227] 10.222.0.83 () {56 vars in 1143 bytes} [Wed Aug 24 11:08:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:33.678965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d7065-d4b8-4b5c-a0b4-e174a609c671 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:33.781587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d7065-d4b8-4b5c-a0b4-e174a609c671 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:33.816630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b10cdf-40c2-4a1d-8b78-d92e0fc0a0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:33.817218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b10cdf-40c2-4a1d-8b78-d92e0fc0a0ea None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:33.817495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b10cdf-40c2-4a1d-8b78-d92e0fc0a0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:33.817771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b10cdf-40c2-4a1d-8b78-d92e0fc0a0ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:33.830425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b10cdf-40c2-4a1d-8b78-d92e0fc0a0ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:33.831892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b10cdf-40c2-4a1d-8b78-d92e0fc0a0ea None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-24b10cdf-40c2-4a1d-8b78-d92e0fc0a0ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:33.846410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:33.847717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:33.848898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:33.848898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:33.848898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:33.848898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 615/1228] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:33.853674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c76af-9f7b-4103-89ed-d314c23147e0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:33.962600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c76af-9f7b-4103-89ed-d314c23147e0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:33.965111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:33.998156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae3b31-0a07-4796-8fd5-d67a0c91ee71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:33.998901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae3b31-0a07-4796-8fd5-d67a0c91ee71 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:33.999181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae3b31-0a07-4796-8fd5-d67a0c91ee71 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:33.999454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae3b31-0a07-4796-8fd5-d67a0c91ee71 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:34.000121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae3b31-0a07-4796-8fd5-d67a0c91ee71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:34.000422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae3b31-0a07-4796-8fd5-d67a0c91ee71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:34.002404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae3b31-0a07-4796-8fd5-d67a0c91ee71 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:34.014250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 614/1229] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:34.020560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b10cdf-40c2-4a1d-8b78-d92e0fc0a0ea None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:34.030170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:34.122962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b10cdf-40c2-4a1d-8b78-d92e0fc0a0ea None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:34.157685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067374cf-60a1-4c1e-893b-8c32ba448f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:34.158346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067374cf-60a1-4c1e-893b-8c32ba448f0a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:34.158643 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067374cf-60a1-4c1e-893b-8c32ba448f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:34.158911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067374cf-60a1-4c1e-893b-8c32ba448f0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:34.165704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067374cf-60a1-4c1e-893b-8c32ba448f0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:34.167167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067374cf-60a1-4c1e-893b-8c32ba448f0a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-067374cf-60a1-4c1e-893b-8c32ba448f0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:08:34.184089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:34.185241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:34.186524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:08:34.186524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:08:34.186524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:08:34.186524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:08:34.186524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:08:34.186524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:08:34.186524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:34.186524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:34.186524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 616/1230] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:34.192456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae3b31-0a07-4796-8fd5-d67a0c91ee71 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:34.293896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae3b31-0a07-4796-8fd5-d67a0c91ee71 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:34.329425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aca42e-486c-47d8-baad-1f2ecc3d8e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:34.330116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aca42e-486c-47d8-baad-1f2ecc3d8e82 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:34.330565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aca42e-486c-47d8-baad-1f2ecc3d8e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:34.330845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aca42e-486c-47d8-baad-1f2ecc3d8e82 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:34.331542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aca42e-486c-47d8-baad-1f2ecc3d8e82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:34.331848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aca42e-486c-47d8-baad-1f2ecc3d8e82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:34.333341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aca42e-486c-47d8-baad-1f2ecc3d8e82 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:34.347739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 615/1231] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:34.353835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067374cf-60a1-4c1e-893b-8c32ba448f0a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:34.425075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:34.461529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067374cf-60a1-4c1e-893b-8c32ba448f0a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:34.496136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbff84-a61a-4296-9914-67281d93dd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:34.496872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbff84-a61a-4296-9914-67281d93dd37 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:34.497154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbff84-a61a-4296-9914-67281d93dd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:34.497439 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbff84-a61a-4296-9914-67281d93dd37 None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/eb76ad667a744126ab00b38c346e9017/roles/7612e1bae5ad4bda9f0675ad732646b4` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:34.514135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbff84-a61a-4296-9914-67281d93dd37 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=eb76ad667a744126ab00b38c346e9017, role_id=7612e1bae5ad4bda9f0675ad732646b4)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:34.515633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbff84-a61a-4296-9914-67281d93dd37 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:34.653347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 617/1232] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:08:34 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/eb76ad667a744126ab00b38c346e9017/roles/7612e1bae5ad4bda9f0675ad732646b4 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:36.598897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f17a0-e160-4a49-ba95-220bad1c4a84 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:36.599770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f17a0-e160-4a49-ba95-220bad1c4a84 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:36.599770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f17a0-e160-4a49-ba95-220bad1c4a84 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:36.600869 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 616/1233] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:36.607001 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c9864-9b40-4bfd-8696-a35c43e53fb5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:36.607493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c9864-9b40-4bfd-8696-a35c43e53fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:36.607584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c9864-9b40-4bfd-8696-a35c43e53fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:36.625412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-170c9864-9b40-4bfd-8696-a35c43e53fb5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:36.626645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-170c9864-9b40-4bfd-8696-a35c43e53fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:36.695379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 618/1234] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:08:36.962747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f17a0-e160-4a49-ba95-220bad1c4a84 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:36.988941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:37.073646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f17a0-e160-4a49-ba95-220bad1c4a84 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:37.112617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54bd69-1fc8-4998-8be9-880d5395817b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:37.113240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54bd69-1fc8-4998-8be9-880d5395817b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:37.113516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54bd69-1fc8-4998-8be9-880d5395817b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:37.113781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54bd69-1fc8-4998-8be9-880d5395817b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:37.125201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54bd69-1fc8-4998-8be9-880d5395817b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3a54bd69-1fc8-4998-8be9-880d5395817b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:08:37.131514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:37.132569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:37.135075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 617/1235] 10.222.0.83 () {56 vars in 1102 bytes} [Wed Aug 24 11:08:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:37.139102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c9864-9b40-4bfd-8696-a35c43e53fb5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:37.188661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:37.243084 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c9864-9b40-4bfd-8696-a35c43e53fb5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:37.278686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c2d7f-75c8-4e52-be9b-3f0130144f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:37.279282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c2d7f-75c8-4e52-be9b-3f0130144f5e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:37.279558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c2d7f-75c8-4e52-be9b-3f0130144f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:37.279827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c2d7f-75c8-4e52-be9b-3f0130144f5e None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:37.280599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c2d7f-75c8-4e52-be9b-3f0130144f5e None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:37.280899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c2d7f-75c8-4e52-be9b-3f0130144f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:37.282391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c2d7f-75c8-4e52-be9b-3f0130144f5e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:37.291548 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 619/1236] 10.222.0.83 () {56 vars in 1103 bytes} [Wed Aug 24 11:08:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:08:37.298111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54bd69-1fc8-4998-8be9-880d5395817b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:37.385955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:37.401991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54bd69-1fc8-4998-8be9-880d5395817b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:37.439262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d2a86-a922-4be4-9fd2-439963d9e540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:37.439940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d2a86-a922-4be4-9fd2-439963d9e540 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:37.440216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d2a86-a922-4be4-9fd2-439963d9e540 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:37.440491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d2a86-a922-4be4-9fd2-439963d9e540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:37.450672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d2a86-a922-4be4-9fd2-439963d9e540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:37.452170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d2a86-a922-4be4-9fd2-439963d9e540 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-252d2a86-a922-4be4-9fd2-439963d9e540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:37.466748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:37.472488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:37.473682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:37.473682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:37.473682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:37.473682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 618/1237] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:37.475287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c2d7f-75c8-4e52-be9b-3f0130144f5e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:37.579641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c2d7f-75c8-4e52-be9b-3f0130144f5e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:37.588680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:37.617803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3924fb-36b4-4c54-9b27-5d77eb46a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:37.618550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3924fb-36b4-4c54-9b27-5d77eb46a18d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:37.618723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3924fb-36b4-4c54-9b27-5d77eb46a18d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:37.618999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3924fb-36b4-4c54-9b27-5d77eb46a18d None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:37.619663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3924fb-36b4-4c54-9b27-5d77eb46a18d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:37.619959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3924fb-36b4-4c54-9b27-5d77eb46a18d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:37.621332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3924fb-36b4-4c54-9b27-5d77eb46a18d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:37.633989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 620/1238] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:37.640041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d2a86-a922-4be4-9fd2-439963d9e540 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:37.748624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d2a86-a922-4be4-9fd2-439963d9e540 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:37.783976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c410e38-8d65-495b-b516-2f9372100a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:37.784604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c410e38-8d65-495b-b516-2f9372100a1d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:37.784887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c410e38-8d65-495b-b516-2f9372100a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:37.785159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c410e38-8d65-495b-b516-2f9372100a1d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:37.787983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:37.793266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c410e38-8d65-495b-b516-2f9372100a1d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:37.794877 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c410e38-8d65-495b-b516-2f9372100a1d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6c410e38-8d65-495b-b516-2f9372100a1d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:08:37.807162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:37.812756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:37.814484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 619/1239] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:37.815083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3924fb-36b4-4c54-9b27-5d77eb46a18d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:37.919885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3924fb-36b4-4c54-9b27-5d77eb46a18d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:37.955459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa7b4e-4cb9-4f7b-810e-b4b5fd8cc41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:37.956062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa7b4e-4cb9-4f7b-810e-b4b5fd8cc41b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:37.956339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa7b4e-4cb9-4f7b-810e-b4b5fd8cc41b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:37.956609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa7b4e-4cb9-4f7b-810e-b4b5fd8cc41b None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:37.957343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa7b4e-4cb9-4f7b-810e-b4b5fd8cc41b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:37.957649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa7b4e-4cb9-4f7b-810e-b4b5fd8cc41b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:37.959109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa7b4e-4cb9-4f7b-810e-b4b5fd8cc41b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:37.977314 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 621/1240] 10.222.0.83 () {56 vars in 1143 bytes} [Wed Aug 24 11:08:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:37.983468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c410e38-8d65-495b-b516-2f9372100a1d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:38.089631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c410e38-8d65-495b-b516-2f9372100a1d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:38.126598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f4d3c-e932-43fa-8d56-d8218ccb5d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:38.127170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f4d3c-e932-43fa-8d56-d8218ccb5d04 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:38.127450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f4d3c-e932-43fa-8d56-d8218ccb5d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:38.127720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f4d3c-e932-43fa-8d56-d8218ccb5d04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:38.193908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:38.200812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f4d3c-e932-43fa-8d56-d8218ccb5d04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:38.205400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f4d3c-e932-43fa-8d56-d8218ccb5d04 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:38.303454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 620/1241] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:08:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:08:38.309897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa7b4e-4cb9-4f7b-810e-b4b5fd8cc41b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:38.339770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:38.414000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa7b4e-4cb9-4f7b-810e-b4b5fd8cc41b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:38.455753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0565c7-b685-49ae-83d6-0563a335ba7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:38.455753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0565c7-b685-49ae-83d6-0563a335ba7f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:38.455753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0565c7-b685-49ae-83d6-0563a335ba7f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:38.455753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0565c7-b685-49ae-83d6-0563a335ba7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:38.465474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0565c7-b685-49ae-83d6-0563a335ba7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:38.466968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0565c7-b685-49ae-83d6-0563a335ba7f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-3a0565c7-b685-49ae-83d6-0563a335ba7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:38.482873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:38.484091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:38.485369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:38.485369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:38.485369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:38.486042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 622/1242] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:38.490325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f4d3c-e932-43fa-8d56-d8218ccb5d04 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:38.588371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:38.594328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f4d3c-e932-43fa-8d56-d8218ccb5d04 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:38.629063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8567c-ccf6-40d4-9723-b19c0bbbef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:38.629820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8567c-ccf6-40d4-9723-b19c0bbbef6d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:38.630181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8567c-ccf6-40d4-9723-b19c0bbbef6d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:38.630560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8567c-ccf6-40d4-9723-b19c0bbbef6d None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:38.631270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8567c-ccf6-40d4-9723-b19c0bbbef6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:38.631618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8567c-ccf6-40d4-9723-b19c0bbbef6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:38.633091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8567c-ccf6-40d4-9723-b19c0bbbef6d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:38.645527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 621/1243] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:38.651496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0565c7-b685-49ae-83d6-0563a335ba7f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:38.731617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:38.754884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0565c7-b685-49ae-83d6-0563a335ba7f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:38.788808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f3e11-88b8-4e04-96a3-5fb2417bb54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:38.789476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f3e11-88b8-4e04-96a3-5fb2417bb54a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:38.789713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f3e11-88b8-4e04-96a3-5fb2417bb54a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:38.790021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f3e11-88b8-4e04-96a3-5fb2417bb54a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:38.796817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f3e11-88b8-4e04-96a3-5fb2417bb54a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:38.798819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f3e11-88b8-4e04-96a3-5fb2417bb54a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-3d1f3e11-88b8-4e04-96a3-5fb2417bb54a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:08:38.810464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:38.811581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:38.812555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:08:38.812555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:08:38.812555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:08:38.812555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:08:38.812555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:08:38.812555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:08:38.812555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:38.812555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:38.812555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 623/1244] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:38.818399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8567c-ccf6-40d4-9723-b19c0bbbef6d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:38.919053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8567c-ccf6-40d4-9723-b19c0bbbef6d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:38.954885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4384b53-62ed-45d3-8692-bdc28b838981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:38.955498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4384b53-62ed-45d3-8692-bdc28b838981 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:38.955779 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4384b53-62ed-45d3-8692-bdc28b838981 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:38.956054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4384b53-62ed-45d3-8692-bdc28b838981 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:38.956755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4384b53-62ed-45d3-8692-bdc28b838981 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:38.957215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4384b53-62ed-45d3-8692-bdc28b838981 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:38.958710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4384b53-62ed-45d3-8692-bdc28b838981 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:38.972061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 622/1245] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:38.980244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f3e11-88b8-4e04-96a3-5fb2417bb54a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:39.083005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f3e11-88b8-4e04-96a3-5fb2417bb54a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:39.133861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83bc8b-e9b3-40b0-a611-8db4a2e8e028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:39.134523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83bc8b-e9b3-40b0-a611-8db4a2e8e028 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:39.134809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83bc8b-e9b3-40b0-a611-8db4a2e8e028 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:39.135080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83bc8b-e9b3-40b0-a611-8db4a2e8e028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:39.135819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83bc8b-e9b3-40b0-a611-8db4a2e8e028 None admin] RBAC: Adding query filter params (role.id=7612e1bae5ad4bda9f0675ad732646b4, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=eb76ad667a744126ab00b38c346e9017) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:39.136126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83bc8b-e9b3-40b0-a611-8db4a2e8e028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:39.137560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83bc8b-e9b3-40b0-a611-8db4a2e8e028 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:39.144569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 624/1246] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:08:38 2022] GET /identity/v3/role_assignments?user.id=eb76ad667a744126ab00b38c346e9017&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=7612e1bae5ad4bda9f0675ad732646b4 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:08:41.180674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f186a8-7f84-437d-b349-1a652a5ed7e9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:41.181274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f186a8-7f84-437d-b349-1a652a5ed7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:41.181274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f186a8-7f84-437d-b349-1a652a5ed7e9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:41.182674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 623/1247] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:41.188454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f01e5-f669-4757-8cce-2e22e5921fbd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:41.188932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f01e5-f669-4757-8cce-2e22e5921fbd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:41.189026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f01e5-f669-4757-8cce-2e22e5921fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:41.195441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:41.208447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9e7f01e5-f669-4757-8cce-2e22e5921fbd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:41.209645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9e7f01e5-f669-4757-8cce-2e22e5921fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:41.280821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 625/1248] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:08:41.553300 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f186a8-7f84-437d-b349-1a652a5ed7e9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:41.572497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:41.656650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f186a8-7f84-437d-b349-1a652a5ed7e9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:41.691755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f4b7b-fc4e-4774-8c8a-52ac7e01ca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:41.692392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f4b7b-fc4e-4774-8c8a-52ac7e01ca69 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:41.692668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f4b7b-fc4e-4774-8c8a-52ac7e01ca69 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:41.692943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f4b7b-fc4e-4774-8c8a-52ac7e01ca69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:41.703149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f4b7b-fc4e-4774-8c8a-52ac7e01ca69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-db5f4b7b-fc4e-4774-8c8a-52ac7e01ca69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:08:41.707876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:41.709009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:41.710104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 624/1249] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:08:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:41.715113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f01e5-f669-4757-8cce-2e22e5921fbd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:41.825665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f01e5-f669-4757-8cce-2e22e5921fbd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:41.858829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:41.866287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3eacd0-a387-4641-97a5-e90f8fcccb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:41.866939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3eacd0-a387-4641-97a5-e90f8fcccb24 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:41.867215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3eacd0-a387-4641-97a5-e90f8fcccb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:41.867484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3eacd0-a387-4641-97a5-e90f8fcccb24 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:41.868215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3eacd0-a387-4641-97a5-e90f8fcccb24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:41.868522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3eacd0-a387-4641-97a5-e90f8fcccb24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:41.869961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3eacd0-a387-4641-97a5-e90f8fcccb24 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:41.879941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 626/1250] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:08:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:41.886354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f4b7b-fc4e-4774-8c8a-52ac7e01ca69 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:41.966965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:41.990831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f4b7b-fc4e-4774-8c8a-52ac7e01ca69 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:42.027129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346395cf-4154-4f37-b6fd-3d725f4a73d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:42.027760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346395cf-4154-4f37-b6fd-3d725f4a73d4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:42.028042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346395cf-4154-4f37-b6fd-3d725f4a73d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:42.028316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346395cf-4154-4f37-b6fd-3d725f4a73d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:42.039567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346395cf-4154-4f37-b6fd-3d725f4a73d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:42.041154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346395cf-4154-4f37-b6fd-3d725f4a73d4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-346395cf-4154-4f37-b6fd-3d725f4a73d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:42.055954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:42.056975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:42.058077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:42.058077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:42.058077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:42.058077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 625/1251] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:08:42.063886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3eacd0-a387-4641-97a5-e90f8fcccb24 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:42.168418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3eacd0-a387-4641-97a5-e90f8fcccb24 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:42.204853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619e10e-1234-47da-b23e-d6172f60648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:42.205582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619e10e-1234-47da-b23e-d6172f60648a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:42.205761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619e10e-1234-47da-b23e-d6172f60648a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:42.206073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619e10e-1234-47da-b23e-d6172f60648a None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:42.206727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619e10e-1234-47da-b23e-d6172f60648a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:42.207033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619e10e-1234-47da-b23e-d6172f60648a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:42.208398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619e10e-1234-47da-b23e-d6172f60648a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:42.220773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 627/1252] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:42.226750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346395cf-4154-4f37-b6fd-3d725f4a73d4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:42.329842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346395cf-4154-4f37-b6fd-3d725f4a73d4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:42.356257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:42.364611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca585722-b45a-43dd-8f32-df331417f856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:42.365251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca585722-b45a-43dd-8f32-df331417f856 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:42.365537 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca585722-b45a-43dd-8f32-df331417f856 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:42.365810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca585722-b45a-43dd-8f32-df331417f856 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:42.373397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca585722-b45a-43dd-8f32-df331417f856 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:42.374876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca585722-b45a-43dd-8f32-df331417f856 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ca585722-b45a-43dd-8f32-df331417f856 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:08:42.386812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:42.388144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:42.389118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 626/1253] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:42.394107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619e10e-1234-47da-b23e-d6172f60648a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:42.455130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:42.499527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619e10e-1234-47da-b23e-d6172f60648a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:42.535358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ae2e2-95b1-450a-9070-0764bfe677aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:42.535964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ae2e2-95b1-450a-9070-0764bfe677aa None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:42.536245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ae2e2-95b1-450a-9070-0764bfe677aa None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:42.536516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ae2e2-95b1-450a-9070-0764bfe677aa None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:42.537275 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ae2e2-95b1-450a-9070-0764bfe677aa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:42.537577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ae2e2-95b1-450a-9070-0764bfe677aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:42.539151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ae2e2-95b1-450a-9070-0764bfe677aa None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:42.558920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 628/1254] 10.222.0.83 () {56 vars in 1143 bytes} [Wed Aug 24 11:08:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:42.565929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca585722-b45a-43dd-8f32-df331417f856 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:42.671199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca585722-b45a-43dd-8f32-df331417f856 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:42.707293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9dc61-7934-46c2-95a1-66d9dadd308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:42.707815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9dc61-7934-46c2-95a1-66d9dadd308d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:42.708094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9dc61-7934-46c2-95a1-66d9dadd308d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:42.708366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9dc61-7934-46c2-95a1-66d9dadd308d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:42.764662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:42.776922 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9dc61-7934-46c2-95a1-66d9dadd308d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:42.778356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9dc61-7934-46c2-95a1-66d9dadd308d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:42.877368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 627/1255] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:08:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:08:42.883493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ae2e2-95b1-450a-9070-0764bfe677aa None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:42.986395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ae2e2-95b1-450a-9070-0764bfe677aa None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:42.999779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:43.020420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff06ed5-f434-4ea6-9075-0bd07445d3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:43.030329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff06ed5-f434-4ea6-9075-0bd07445d3eb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:43.030329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff06ed5-f434-4ea6-9075-0bd07445d3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:43.030329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff06ed5-f434-4ea6-9075-0bd07445d3eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:43.033032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff06ed5-f434-4ea6-9075-0bd07445d3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:43.034534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff06ed5-f434-4ea6-9075-0bd07445d3eb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8ff06ed5-f434-4ea6-9075-0bd07445d3eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:43.050134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:43.051346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:43.052630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:43.052630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:43.052630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:43.052630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 629/1256] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:43.057486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9dc61-7934-46c2-95a1-66d9dadd308d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:43.148054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:43.162990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9dc61-7934-46c2-95a1-66d9dadd308d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:43.201002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777c564-4675-48a3-9e2f-3db491f16c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:43.201612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777c564-4675-48a3-9e2f-3db491f16c34 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:43.201911 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777c564-4675-48a3-9e2f-3db491f16c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:43.202189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777c564-4675-48a3-9e2f-3db491f16c34 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:43.202842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777c564-4675-48a3-9e2f-3db491f16c34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:43.203148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777c564-4675-48a3-9e2f-3db491f16c34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:43.205071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777c564-4675-48a3-9e2f-3db491f16c34 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:43.216632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 628/1257] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:43.223145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff06ed5-f434-4ea6-9075-0bd07445d3eb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:43.324768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff06ed5-f434-4ea6-9075-0bd07445d3eb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:43.359791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9fced-e683-4416-9fdc-4460ec574140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:43.360429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9fced-e683-4416-9fdc-4460ec574140 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:43.360702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9fced-e683-4416-9fdc-4460ec574140 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:43.360975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9fced-e683-4416-9fdc-4460ec574140 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:43.367754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9fced-e683-4416-9fdc-4460ec574140 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:43.369211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9fced-e683-4416-9fdc-4460ec574140 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-93a9fced-e683-4416-9fdc-4460ec574140 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:08:43.380640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:43.383893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:43.384936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:08:43.384936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:08:43.384936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:08:43.384936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:08:43.384936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:08:43.384936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:08:43.384936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:43.384936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:43.384936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 630/1258] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:08:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:43.387754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777c564-4675-48a3-9e2f-3db491f16c34 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:43.492943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777c564-4675-48a3-9e2f-3db491f16c34 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:43.527706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec743d5f-7783-4304-8189-1c9306d942e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:43.528312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec743d5f-7783-4304-8189-1c9306d942e3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:43.528590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec743d5f-7783-4304-8189-1c9306d942e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:43.528860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec743d5f-7783-4304-8189-1c9306d942e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:43.529556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec743d5f-7783-4304-8189-1c9306d942e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:43.529862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec743d5f-7783-4304-8189-1c9306d942e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:43.531439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec743d5f-7783-4304-8189-1c9306d942e3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:43.544318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 629/1259] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:08:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:43.551531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9fced-e683-4416-9fdc-4460ec574140 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:43.579931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:43.656471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9fced-e683-4416-9fdc-4460ec574140 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:43.695773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ac2ef-a807-4374-8d12-a2798969f43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:43.696364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ac2ef-a807-4374-8d12-a2798969f43b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:43.696640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ac2ef-a807-4374-8d12-a2798969f43b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:43.696910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ac2ef-a807-4374-8d12-a2798969f43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:43.697681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ac2ef-a807-4374-8d12-a2798969f43b None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=eb76ad667a744126ab00b38c346e9017) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:43.698220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ac2ef-a807-4374-8d12-a2798969f43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:43.700112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ac2ef-a807-4374-8d12-a2798969f43b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:43.706377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 631/1260] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:08:43 2022] GET /identity/v3/role_assignments?user.id=eb76ad667a744126ab00b38c346e9017&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:45.615931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c42da-668f-4c60-beda-c199d992a106 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:45.616513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c42da-668f-4c60-beda-c199d992a106 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:45.616513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c42da-668f-4c60-beda-c199d992a106 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:45.618028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 630/1261] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:08:45.623603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5897e-6a37-4f4a-8bac-b903ee4afb16 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:45.624109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5897e-6a37-4f4a-8bac-b903ee4afb16 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:45.624254 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5897e-6a37-4f4a-8bac-b903ee4afb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:45.642384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0ba5897e-6a37-4f4a-8bac-b903ee4afb16 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:45.643560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0ba5897e-6a37-4f4a-8bac-b903ee4afb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:45.723238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 632/1262] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:08:45.997076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c42da-668f-4c60-beda-c199d992a106 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:46.008603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:46.102590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c42da-668f-4c60-beda-c199d992a106 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:46.140482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779f42-be80-46ea-9d07-53300ea96aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:46.141143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779f42-be80-46ea-9d07-53300ea96aef None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:46.141418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779f42-be80-46ea-9d07-53300ea96aef None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:46.141691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779f42-be80-46ea-9d07-53300ea96aef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:46.152481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f779f42-be80-46ea-9d07-53300ea96aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-9f779f42-be80-46ea-9d07-53300ea96aef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:08:46.157251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:46.158399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:46.159238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 631/1263] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:08:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:46.164992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5897e-6a37-4f4a-8bac-b903ee4afb16 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:46.184807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:46.268599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5897e-6a37-4f4a-8bac-b903ee4afb16 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:46.303846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510e8f-b02a-4019-8f23-261394834e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:46.304450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510e8f-b02a-4019-8f23-261394834e43 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:46.304727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510e8f-b02a-4019-8f23-261394834e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:46.304997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510e8f-b02a-4019-8f23-261394834e43 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:46.305665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510e8f-b02a-4019-8f23-261394834e43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:46.306007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510e8f-b02a-4019-8f23-261394834e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:46.307388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510e8f-b02a-4019-8f23-261394834e43 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:46.316422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 633/1264] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:08:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:08:46.322753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779f42-be80-46ea-9d07-53300ea96aef None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:46.395611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:46.430981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779f42-be80-46ea-9d07-53300ea96aef None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:46.465469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12372e36-eae0-4003-a5fd-26826a980d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:46.466167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12372e36-eae0-4003-a5fd-26826a980d30 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:46.466522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12372e36-eae0-4003-a5fd-26826a980d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:46.468434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12372e36-eae0-4003-a5fd-26826a980d30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:46.478293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12372e36-eae0-4003-a5fd-26826a980d30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:46.479761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12372e36-eae0-4003-a5fd-26826a980d30 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-12372e36-eae0-4003-a5fd-26826a980d30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:46.494924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:46.496155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:46.497297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:46.497297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:46.497297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:46.497297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 632/1265] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:46.502663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510e8f-b02a-4019-8f23-261394834e43 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:46.589680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:46.611131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510e8f-b02a-4019-8f23-261394834e43 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:46.644320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f532abd-fe6c-4441-b32e-420494de0cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:46.644992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f532abd-fe6c-4441-b32e-420494de0cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:46.645194 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f532abd-fe6c-4441-b32e-420494de0cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:46.645495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f532abd-fe6c-4441-b32e-420494de0cf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:46.646174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f532abd-fe6c-4441-b32e-420494de0cf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:46.646480 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f532abd-fe6c-4441-b32e-420494de0cf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:46.647842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f532abd-fe6c-4441-b32e-420494de0cf1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:46.660655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 634/1266] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:46.666663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12372e36-eae0-4003-a5fd-26826a980d30 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:46.770761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12372e36-eae0-4003-a5fd-26826a980d30 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:46.791154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:46.806123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac851e-539e-4fd9-a3e0-29c3a008a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:46.806753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac851e-539e-4fd9-a3e0-29c3a008a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:46.807030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac851e-539e-4fd9-a3e0-29c3a008a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:46.807308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac851e-539e-4fd9-a3e0-29c3a008a0c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:46.815722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac851e-539e-4fd9-a3e0-29c3a008a0c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:46.817183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac851e-539e-4fd9-a3e0-29c3a008a0c2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-0eac851e-539e-4fd9-a3e0-29c3a008a0c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:08:46.829025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:46.831588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:46.832782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 633/1267] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:46.836347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f532abd-fe6c-4441-b32e-420494de0cf1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:46.938900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f532abd-fe6c-4441-b32e-420494de0cf1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:46.973856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c79a6-dab5-4783-b874-1d2c36730054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:46.988265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c79a6-dab5-4783-b874-1d2c36730054 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:46.988265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c79a6-dab5-4783-b874-1d2c36730054 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:46.988265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c79a6-dab5-4783-b874-1d2c36730054 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:46.988265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c79a6-dab5-4783-b874-1d2c36730054 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:46.988265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c79a6-dab5-4783-b874-1d2c36730054 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:46.988265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c79a6-dab5-4783-b874-1d2c36730054 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:46.988776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:46.994889 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 635/1268] 10.222.0.83 () {56 vars in 1143 bytes} [Wed Aug 24 11:08:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:47.000703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac851e-539e-4fd9-a3e0-29c3a008a0c2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:47.104866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac851e-539e-4fd9-a3e0-29c3a008a0c2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:47.140315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96f68d-d625-40cc-9dbb-698b38536cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:47.141456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96f68d-d625-40cc-9dbb-698b38536cbc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:47.141736 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96f68d-d625-40cc-9dbb-698b38536cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:47.142083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96f68d-d625-40cc-9dbb-698b38536cbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:47.152921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96f68d-d625-40cc-9dbb-698b38536cbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:47.154443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96f68d-d625-40cc-9dbb-698b38536cbc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-0b96f68d-d625-40cc-9dbb-698b38536cbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:47.169802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:47.170860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:47.172082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:47.172082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:47.172082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:47.172082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 634/1269] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:47.177500 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c79a6-dab5-4783-b874-1d2c36730054 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:47.285494 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c79a6-dab5-4783-b874-1d2c36730054 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:47.321735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb559b-adbc-4dad-a68b-e697e98ada26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:47.322382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb559b-adbc-4dad-a68b-e697e98ada26 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:47.322650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb559b-adbc-4dad-a68b-e697e98ada26 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:47.322921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb559b-adbc-4dad-a68b-e697e98ada26 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:47.323570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb559b-adbc-4dad-a68b-e697e98ada26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:47.323922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb559b-adbc-4dad-a68b-e697e98ada26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:47.325582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb559b-adbc-4dad-a68b-e697e98ada26 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:47.338146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 636/1270] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:47.343793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96f68d-d625-40cc-9dbb-698b38536cbc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:47.376561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:47.448603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96f68d-d625-40cc-9dbb-698b38536cbc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:47.484035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4329f-2449-4faf-8f68-52d6af975471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:47.484689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4329f-2449-4faf-8f68-52d6af975471 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:47.485175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4329f-2449-4faf-8f68-52d6af975471 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:47.485400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4329f-2449-4faf-8f68-52d6af975471 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:47.492449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4329f-2449-4faf-8f68-52d6af975471 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:47.493924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4329f-2449-4faf-8f68-52d6af975471 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7bb4329f-2449-4faf-8f68-52d6af975471 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:08:47.506189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:47.507274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:47.508273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:08:47.508273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:08:47.508273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:08:47.508273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:08:47.508273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:08:47.508273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:08:47.508273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:47.508273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:47.508273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 635/1271] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:08:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:47.513656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb559b-adbc-4dad-a68b-e697e98ada26 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:47.580971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:47.623463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb559b-adbc-4dad-a68b-e697e98ada26 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:47.658238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b46202-a7d4-4524-bae6-803aa0ef728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:47.658880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b46202-a7d4-4524-bae6-803aa0ef728a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:47.659102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b46202-a7d4-4524-bae6-803aa0ef728a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:47.659375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b46202-a7d4-4524-bae6-803aa0ef728a None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:47.660072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b46202-a7d4-4524-bae6-803aa0ef728a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:47.660379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b46202-a7d4-4524-bae6-803aa0ef728a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:47.661803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b46202-a7d4-4524-bae6-803aa0ef728a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:47.674085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 637/1272] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:08:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:47.680515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4329f-2449-4faf-8f68-52d6af975471 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:47.783099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:47.792472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4329f-2449-4faf-8f68-52d6af975471 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:47.827823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43911f35-3217-44cf-9478-8bc6b2a27471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:47.829269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911f35-3217-44cf-9478-8bc6b2a27471 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:47.829269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911f35-3217-44cf-9478-8bc6b2a27471 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:47.829575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911f35-3217-44cf-9478-8bc6b2a27471 None admin] PATH_INFO: `/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/eb76ad667a744126ab00b38c346e9017/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:47.831595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911f35-3217-44cf-9478-8bc6b2a27471 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd7278f8c3f04cd391e6ffda9a5c8217, user_id=eb76ad667a744126ab00b38c346e9017, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:47.832985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911f35-3217-44cf-9478-8bc6b2a27471 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:47.913605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 636/1273] 10.222.0.83 () {58 vars in 1439 bytes} [Wed Aug 24 11:08:47 2022] PUT /identity/v3/projects/cd7278f8c3f04cd391e6ffda9a5c8217/users/eb76ad667a744126ab00b38c346e9017/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:08:49.858217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a25f5a-a406-4fd8-a592-e5f1cd0dcfe1 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:49.858748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a25f5a-a406-4fd8-a592-e5f1cd0dcfe1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:49.858748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a25f5a-a406-4fd8-a592-e5f1cd0dcfe1 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:49.860601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 638/1274] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:49.866409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe9d27-e737-45e0-83d9-8358d6d8ce2e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:49.866847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe9d27-e737-45e0-83d9-8358d6d8ce2e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:49.866961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe9d27-e737-45e0-83d9-8358d6d8ce2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:49.886404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8efe9d27-e737-45e0-83d9-8358d6d8ce2e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:49.887614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8efe9d27-e737-45e0-83d9-8358d6d8ce2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:49.961346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 637/1275] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:08:50.234767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a25f5a-a406-4fd8-a592-e5f1cd0dcfe1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:50.342848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a25f5a-a406-4fd8-a592-e5f1cd0dcfe1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:50.360278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:50.377171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bbe77-6d72-4b58-af2b-d2c8c637df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:50.377901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bbe77-6d72-4b58-af2b-d2c8c637df67 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:50.378169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bbe77-6d72-4b58-af2b-d2c8c637df67 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:50.378435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bbe77-6d72-4b58-af2b-d2c8c637df67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:50.388619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bbe77-6d72-4b58-af2b-d2c8c637df67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-598bbe77-6d72-4b58-af2b-d2c8c637df67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:08:50.393419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:50.395293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:50.396397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 639/1276] 10.222.0.83 () {56 vars in 1096 bytes} [Wed Aug 24 11:08:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:50.402126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe9d27-e737-45e0-83d9-8358d6d8ce2e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:50.485335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:50.504038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe9d27-e737-45e0-83d9-8358d6d8ce2e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:50.539601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5b182-f346-465b-97af-1a9fcc161ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:50.540210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5b182-f346-465b-97af-1a9fcc161ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:50.540487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5b182-f346-465b-97af-1a9fcc161ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:50.540757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5b182-f346-465b-97af-1a9fcc161ec9 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:50.541430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5b182-f346-465b-97af-1a9fcc161ec9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:50.541732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5b182-f346-465b-97af-1a9fcc161ec9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:50.543280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5b182-f346-465b-97af-1a9fcc161ec9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:50.551184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 638/1277] 10.222.0.83 () {56 vars in 1099 bytes} [Wed Aug 24 11:08:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:50.557288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bbe77-6d72-4b58-af2b-d2c8c637df67 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:50.661273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bbe77-6d72-4b58-af2b-d2c8c637df67 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:50.697637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98929c41-f3a7-4260-acd6-2855d060a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:50.698325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98929c41-f3a7-4260-acd6-2855d060a93e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:50.698605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98929c41-f3a7-4260-acd6-2855d060a93e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:50.698875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98929c41-f3a7-4260-acd6-2855d060a93e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:50.709299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98929c41-f3a7-4260-acd6-2855d060a93e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:50.710788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98929c41-f3a7-4260-acd6-2855d060a93e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-98929c41-f3a7-4260-acd6-2855d060a93e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:50.725485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:50.726618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:50.727745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:50.727745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:50.727745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:50.727745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 640/1278] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:50.732770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5b182-f346-465b-97af-1a9fcc161ec9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:50.834382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5b182-f346-465b-97af-1a9fcc161ec9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:50.869228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3f56d-6093-40f9-aead-b3811ce9648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:50.869938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3f56d-6093-40f9-aead-b3811ce9648f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:50.870136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3f56d-6093-40f9-aead-b3811ce9648f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:50.870409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3f56d-6093-40f9-aead-b3811ce9648f None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:50.871058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3f56d-6093-40f9-aead-b3811ce9648f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:50.871490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3f56d-6093-40f9-aead-b3811ce9648f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:50.872886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3f56d-6093-40f9-aead-b3811ce9648f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:50.886018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 639/1279] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:50.894657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98929c41-f3a7-4260-acd6-2855d060a93e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:50.928981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:51.003209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98929c41-f3a7-4260-acd6-2855d060a93e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:51.036444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09d23b-99e4-4bf9-abab-5fab28968ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:51.037064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09d23b-99e4-4bf9-abab-5fab28968ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:51.037337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09d23b-99e4-4bf9-abab-5fab28968ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:51.037681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09d23b-99e4-4bf9-abab-5fab28968ff8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:51.045952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09d23b-99e4-4bf9-abab-5fab28968ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:51.047417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09d23b-99e4-4bf9-abab-5fab28968ff8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-fa09d23b-99e4-4bf9-abab-5fab28968ff8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:08:51.059114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:51.060208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:51.061539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 641/1280] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:51.066781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3f56d-6093-40f9-aead-b3811ce9648f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:51.070605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:51.174514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3f56d-6093-40f9-aead-b3811ce9648f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:51.213242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b261e-b4cf-49d3-8192-bbdafb63d21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:51.213854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b261e-b4cf-49d3-8192-bbdafb63d21e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:51.214159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b261e-b4cf-49d3-8192-bbdafb63d21e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:51.214434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b261e-b4cf-49d3-8192-bbdafb63d21e None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:51.215170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b261e-b4cf-49d3-8192-bbdafb63d21e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:51.215479 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b261e-b4cf-49d3-8192-bbdafb63d21e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:51.216956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b261e-b4cf-49d3-8192-bbdafb63d21e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:51.235635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 640/1281] 10.222.0.83 () {56 vars in 1143 bytes} [Wed Aug 24 11:08:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:51.241670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09d23b-99e4-4bf9-abab-5fab28968ff8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:51.343729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:51.353322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09d23b-99e4-4bf9-abab-5fab28968ff8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:51.387333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972a984-f545-4883-b9db-693af2355832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:51.387927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972a984-f545-4883-b9db-693af2355832 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:51.388227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972a984-f545-4883-b9db-693af2355832 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:51.388499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972a984-f545-4883-b9db-693af2355832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:51.459878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972a984-f545-4883-b9db-693af2355832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:51.461259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972a984-f545-4883-b9db-693af2355832 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:51.552513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:51.563055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 642/1282] 10.222.0.83 () {58 vars in 1282 bytes} [Wed Aug 24 11:08:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:08:51.569585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b261e-b4cf-49d3-8192-bbdafb63d21e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:51.632770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:51.673992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b261e-b4cf-49d3-8192-bbdafb63d21e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:51.708438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5073b-4ead-4a0d-bb72-a7fda13c98b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:51.709061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5073b-4ead-4a0d-bb72-a7fda13c98b2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:51.709331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5073b-4ead-4a0d-bb72-a7fda13c98b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:51.709598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5073b-4ead-4a0d-bb72-a7fda13c98b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:51.720277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5073b-4ead-4a0d-bb72-a7fda13c98b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:51.721769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5073b-4ead-4a0d-bb72-a7fda13c98b2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-58e5073b-4ead-4a0d-bb72-a7fda13c98b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:08:51.736842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:51.739214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:08:51.740135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:08:51.740135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:08:51.740135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:51.740135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 641/1283] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:51.743837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972a984-f545-4883-b9db-693af2355832 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:51.851830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972a984-f545-4883-b9db-693af2355832 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:51.887670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b47fb-838e-4a1c-8592-178b28ea58f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:51.888275 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b47fb-838e-4a1c-8592-178b28ea58f8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:51.888550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b47fb-838e-4a1c-8592-178b28ea58f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:51.888817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b47fb-838e-4a1c-8592-178b28ea58f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:51.889467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b47fb-838e-4a1c-8592-178b28ea58f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:51.889758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b47fb-838e-4a1c-8592-178b28ea58f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:51.891147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b47fb-838e-4a1c-8592-178b28ea58f8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:51.904620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 643/1284] 10.222.0.83 () {56 vars in 1109 bytes} [Wed Aug 24 11:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:51.910936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5073b-4ead-4a0d-bb72-a7fda13c98b2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:52.023451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5073b-4ead-4a0d-bb72-a7fda13c98b2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.037739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:52.058522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad49716-2aff-433d-9668-7deb28bb63c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.059139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad49716-2aff-433d-9668-7deb28bb63c9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.059416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad49716-2aff-433d-9668-7deb28bb63c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.059683 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad49716-2aff-433d-9668-7deb28bb63c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.066260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad49716-2aff-433d-9668-7deb28bb63c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.067705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad49716-2aff-433d-9668-7deb28bb63c9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3ad49716-2aff-433d-9668-7deb28bb63c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:08:52.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:52.081205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:52.082207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:08:52.082207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:08:52.082207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:08:52.082207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:08:52.082207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:08:52.082207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:08:52.082207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:08:52.082207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Aug 24 11:08:52.082207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 642/1285] 10.222.0.83 () {56 vars in 1111 bytes} [Wed Aug 24 11:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:52.088632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b47fb-838e-4a1c-8592-178b28ea58f8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:52.140873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:52.196769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b47fb-838e-4a1c-8592-178b28ea58f8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.232660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ff6cc-f322-4d7b-87db-ee4384cb1bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.233353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ff6cc-f322-4d7b-87db-ee4384cb1bfe None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.233553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ff6cc-f322-4d7b-87db-ee4384cb1bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.233823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ff6cc-f322-4d7b-87db-ee4384cb1bfe None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.234574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ff6cc-f322-4d7b-87db-ee4384cb1bfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:52.234880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ff6cc-f322-4d7b-87db-ee4384cb1bfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.236304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ff6cc-f322-4d7b-87db-ee4384cb1bfe None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:52.252238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 644/1286] 10.222.0.83 () {56 vars in 1148 bytes} [Wed Aug 24 11:08:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:52.258497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad49716-2aff-433d-9668-7deb28bb63c9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:52.366543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad49716-2aff-433d-9668-7deb28bb63c9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:52.400807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e6baf-e9bd-4c5e-943b-9b837cfa7387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:52.401412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e6baf-e9bd-4c5e-943b-9b837cfa7387 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:52.401718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e6baf-e9bd-4c5e-943b-9b837cfa7387 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:52.402024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e6baf-e9bd-4c5e-943b-9b837cfa7387 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:52.402768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e6baf-e9bd-4c5e-943b-9b837cfa7387 None admin] RBAC: Adding query filter params (role.id=80b00b8da5b24da8bce7c6fe885050f8, scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217, user.id=eb76ad667a744126ab00b38c346e9017) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:52.403072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e6baf-e9bd-4c5e-943b-9b837cfa7387 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:52.404546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e6baf-e9bd-4c5e-943b-9b837cfa7387 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:52.411468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 643/1287] 10.222.0.83 () {56 vars in 1374 bytes} [Wed Aug 24 11:08:52 2022] GET /identity/v3/role_assignments?user.id=eb76ad667a744126ab00b38c346e9017&scope.project.id=cd7278f8c3f04cd391e6ffda9a5c8217&role.id=80b00b8da5b24da8bce7c6fe885050f8 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:54.371031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706c2eb-46aa-4ca7-a0ee-6ae914e865a1 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:54.371616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706c2eb-46aa-4ca7-a0ee-6ae914e865a1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:54.371616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706c2eb-46aa-4ca7-a0ee-6ae914e865a1 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:54.372900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 645/1288] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:54.378475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0145f82-186e-4c2e-b516-25ea942cb3ad None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:54.381774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0145f82-186e-4c2e-b516-25ea942cb3ad None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:54.381774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0145f82-186e-4c2e-b516-25ea942cb3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:54.398489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c0145f82-186e-4c2e-b516-25ea942cb3ad None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:54.399776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c0145f82-186e-4c2e-b516-25ea942cb3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:54.431834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:54.470591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 644/1289] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:08:54.772738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706c2eb-46aa-4ca7-a0ee-6ae914e865a1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:54.878506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706c2eb-46aa-4ca7-a0ee-6ae914e865a1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:54.890332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:54.913848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e635d6d-615f-44fc-a5d1-07006b4b9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:54.914553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e635d6d-615f-44fc-a5d1-07006b4b9537 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:54.914820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e635d6d-615f-44fc-a5d1-07006b4b9537 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:54.915092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e635d6d-615f-44fc-a5d1-07006b4b9537 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:54.915674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e635d6d-615f-44fc-a5d1-07006b4b9537 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:54.917044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e635d6d-615f-44fc-a5d1-07006b4b9537 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8e635d6d-615f-44fc-a5d1-07006b4b9537 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:08:54.926593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:08:54.927916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:08:54.929578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 646/1290] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:08:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:08:54.935006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0145f82-186e-4c2e-b516-25ea942cb3ad None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:55.053099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0145f82-186e-4c2e-b516-25ea942cb3ad None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:55.099047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d0db8-8d51-4c9f-a315-23456347280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:55.099646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d0db8-8d51-4c9f-a315-23456347280e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:55.099924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d0db8-8d51-4c9f-a315-23456347280e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:55.100191 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d0db8-8d51-4c9f-a315-23456347280e None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:55.100830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d0db8-8d51-4c9f-a315-23456347280e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:55.101131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d0db8-8d51-4c9f-a315-23456347280e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:55.102521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d0db8-8d51-4c9f-a315-23456347280e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:55.107429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 645/1291] 10.222.0.83 () {56 vars in 1116 bytes} [Wed Aug 24 11:08:54 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:55.113441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e635d6d-615f-44fc-a5d1-07006b4b9537 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:55.220067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e635d6d-615f-44fc-a5d1-07006b4b9537 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:55.255082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba484d-4bcf-40fb-a961-cd4d05bea9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:55.255691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba484d-4bcf-40fb-a961-cd4d05bea9df None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:55.255965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba484d-4bcf-40fb-a961-cd4d05bea9df None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:55.256375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba484d-4bcf-40fb-a961-cd4d05bea9df None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:55.257157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba484d-4bcf-40fb-a961-cd4d05bea9df None admin] RBAC: Adding query filter params (type=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:08:55.257473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba484d-4bcf-40fb-a961-cd4d05bea9df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:55.258891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba484d-4bcf-40fb-a961-cd4d05bea9df None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:55.263803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 647/1292] 10.222.0.83 () {56 vars in 1116 bytes} [Wed Aug 24 11:08:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:08:57.207904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580b90a-f985-4a36-b2aa-0ded0af3dba9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:57.208434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580b90a-f985-4a36-b2aa-0ded0af3dba9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:57.208527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580b90a-f985-4a36-b2aa-0ded0af3dba9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:57.209961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 646/1293] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:57.215647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abeca5-6d92-4384-884e-489d8b2a30de None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:57.216107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abeca5-6d92-4384-884e-489d8b2a30de None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:57.216204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abeca5-6d92-4384-884e-489d8b2a30de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:57.235146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-70abeca5-6d92-4384-884e-489d8b2a30de None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:57.236413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-70abeca5-6d92-4384-884e-489d8b2a30de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:57.269339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:57.308309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 648/1294] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:08:57.575089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580b90a-f985-4a36-b2aa-0ded0af3dba9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:08:57.610061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:08:57.681434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580b90a-f985-4a36-b2aa-0ded0af3dba9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:08:57.717645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ef7e0-1dbd-4034-9a8c-ba79d67ec66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:08:57.718276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ef7e0-1dbd-4034-9a8c-ba79d67ec66c None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:57.718549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ef7e0-1dbd-4034-9a8c-ba79d67ec66c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:57.718845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ef7e0-1dbd-4034-9a8c-ba79d67ec66c None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:57.719490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ef7e0-1dbd-4034-9a8c-ba79d67ec66c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:08:57.720853 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ef7e0-1dbd-4034-9a8c-ba79d67ec66c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:08:57.814381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 647/1295] 10.222.0.83 () {60 vars in 1141 bytes} [Wed Aug 24 11:08:57 2022] POST /identity/v3/services => generated 246 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:08:59.756242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ab1c9-9fbb-44ae-beab-00d3e13d9e76 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:59.756734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ab1c9-9fbb-44ae-beab-00d3e13d9e76 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:59.756734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ab1c9-9fbb-44ae-beab-00d3e13d9e76 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:59.758166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 649/1296] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:08:59.764688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0cb58-c0e7-48df-8da4-9fdf927c6cef None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:08:59.765221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0cb58-c0e7-48df-8da4-9fdf927c6cef None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:08:59.767102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0cb58-c0e7-48df-8da4-9fdf927c6cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:08:59.788690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-65b0cb58-c0e7-48df-8da4-9fdf927c6cef None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:08:59.795936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-65b0cb58-c0e7-48df-8da4-9fdf927c6cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:08:59.869158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 648/1297] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:08:59 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:09:00.154188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ab1c9-9fbb-44ae-beab-00d3e13d9e76 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:00.258821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ab1c9-9fbb-44ae-beab-00d3e13d9e76 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:00.293831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179c77e-d8d6-4119-bde7-264d3b67b6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:00.294511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179c77e-d8d6-4119-bde7-264d3b67b6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:00.294789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179c77e-d8d6-4119-bde7-264d3b67b6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:00.295060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179c77e-d8d6-4119-bde7-264d3b67b6f0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:00.295659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179c77e-d8d6-4119-bde7-264d3b67b6f0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:00.297539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179c77e-d8d6-4119-bde7-264d3b67b6f0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a179c77e-d8d6-4119-bde7-264d3b67b6f0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:09:00.307168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:09:00.308413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:09:00.309623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 650/1298] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:09:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:09:00.315365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0cb58-c0e7-48df-8da4-9fdf927c6cef None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:00.336837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:00.428131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0cb58-c0e7-48df-8da4-9fdf927c6cef None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:00.464662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4422c-6d5f-4a43-9d8c-5c738e09ab41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:00.465270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4422c-6d5f-4a43-9d8c-5c738e09ab41 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:00.465551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4422c-6d5f-4a43-9d8c-5c738e09ab41 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:00.465822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4422c-6d5f-4a43-9d8c-5c738e09ab41 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:00.466490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4422c-6d5f-4a43-9d8c-5c738e09ab41 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:00.466844 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4422c-6d5f-4a43-9d8c-5c738e09ab41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:00.468221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4422c-6d5f-4a43-9d8c-5c738e09ab41 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:00.473995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 649/1299] 10.222.0.83 () {56 vars in 1116 bytes} [Wed Aug 24 11:09:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:09:00.480052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179c77e-d8d6-4119-bde7-264d3b67b6f0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:00.512298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:00.581362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179c77e-d8d6-4119-bde7-264d3b67b6f0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:00.618092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9fb2f-f471-4b59-a808-ebc98bf31d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:00.618697 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9fb2f-f471-4b59-a808-ebc98bf31d5e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:00.618975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9fb2f-f471-4b59-a808-ebc98bf31d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:00.619247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9fb2f-f471-4b59-a808-ebc98bf31d5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:00.619947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9fb2f-f471-4b59-a808-ebc98bf31d5e None admin] RBAC: Adding query filter params (interface=public, service_id=514f1c22bc184b52b23a95694e1dcec3, region_id=RegionOne) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:00.620251 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9fb2f-f471-4b59-a808-ebc98bf31d5e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:00.621613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9fb2f-f471-4b59-a808-ebc98bf31d5e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:00.626792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 651/1300] 10.222.0.83 () {56 vars in 1251 bytes} [Wed Aug 24 11:09:00 2022] GET /identity/v3/endpoints?service_id=514f1c22bc184b52b23a95694e1dcec3&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:09:00.632694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4422c-6d5f-4a43-9d8c-5c738e09ab41 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:00.716856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:00.741546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4422c-6d5f-4a43-9d8c-5c738e09ab41 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:00.777099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f72bb9-3227-4aaf-a4a9-f8fef1dc74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:00.777993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72bb9-3227-4aaf-a4a9-f8fef1dc74aa None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:00.778357 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72bb9-3227-4aaf-a4a9-f8fef1dc74aa None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:00.778850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72bb9-3227-4aaf-a4a9-f8fef1dc74aa None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:00.779494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f72bb9-3227-4aaf-a4a9-f8fef1dc74aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:00.781863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f72bb9-3227-4aaf-a4a9-f8fef1dc74aa None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:00.788902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 650/1301] 10.222.0.83 () {56 vars in 1087 bytes} [Wed Aug 24 11:09:00 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:09:02.701099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab0ad8-bcd7-47df-b525-50b9da78f580 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:02.701669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab0ad8-bcd7-47df-b525-50b9da78f580 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:02.701669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab0ad8-bcd7-47df-b525-50b9da78f580 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:02.702964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 652/1302] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:09:02.708845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd5fdb-085f-4d25-997f-95842443a4fe None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:02.709317 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd5fdb-085f-4d25-997f-95842443a4fe None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:02.709418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd5fdb-085f-4d25-997f-95842443a4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:02.727665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1dcd5fdb-085f-4d25-997f-95842443a4fe None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:09:02.728865 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1dcd5fdb-085f-4d25-997f-95842443a4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:09:02.801954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 651/1303] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:09:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:09:03.073422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab0ad8-bcd7-47df-b525-50b9da78f580 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:03.175114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:03.184441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab0ad8-bcd7-47df-b525-50b9da78f580 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:03.220141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac39d13-375e-439e-8355-e659ee03fc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:03.223936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac39d13-375e-439e-8355-e659ee03fc24 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:03.223936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac39d13-375e-439e-8355-e659ee03fc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:03.223936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac39d13-375e-439e-8355-e659ee03fc24 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:03.223936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac39d13-375e-439e-8355-e659ee03fc24 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:03.223936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac39d13-375e-439e-8355-e659ee03fc24 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-fac39d13-375e-439e-8355-e659ee03fc24 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:09:03.231813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:09:03.233111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Aug 24 11:09:03.234988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 653/1304] 10.222.0.83 () {56 vars in 1117 bytes} [Wed Aug 24 11:09:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:09:03.239587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd5fdb-085f-4d25-997f-95842443a4fe None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:03.319791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:03.343509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd5fdb-085f-4d25-997f-95842443a4fe None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:03.377912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be832b-7c5e-4436-ab6d-0c5adccdf9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:03.378524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be832b-7c5e-4436-ab6d-0c5adccdf9fc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:03.378806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be832b-7c5e-4436-ab6d-0c5adccdf9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:03.379079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be832b-7c5e-4436-ab6d-0c5adccdf9fc None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:03.379715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be832b-7c5e-4436-ab6d-0c5adccdf9fc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:09:03.380018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be832b-7c5e-4436-ab6d-0c5adccdf9fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:03.381393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be832b-7c5e-4436-ab6d-0c5adccdf9fc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:03.386687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 652/1305] 10.222.0.83 () {56 vars in 1116 bytes} [Wed Aug 24 11:09:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:09:03.392747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac39d13-375e-439e-8355-e659ee03fc24 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:09:03.512005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac39d13-375e-439e-8355-e659ee03fc24 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:09:03.546543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4554e-adeb-414e-9498-0857b2a10178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:09:03.547804 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4554e-adeb-414e-9498-0857b2a10178 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:09:03.548119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4554e-adeb-414e-9498-0857b2a10178 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:09:03.548390 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4554e-adeb-414e-9498-0857b2a10178 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:09:03.549055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4554e-adeb-414e-9498-0857b2a10178 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:09:03.550472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4554e-adeb-414e-9498-0857b2a10178 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:09:03.673819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 654/1306] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:09:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:09:13.435849 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=30 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:09:13.666108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:13.642124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936edd4-301a-4c44-9221-cc8bad851984 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:13.642710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936edd4-301a-4c44-9221-cc8bad851984 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:13.642710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936edd4-301a-4c44-9221-cc8bad851984 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:13.644016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 653/1307] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:13.650325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe6873-2d84-4c65-a1b7-82e8d11bf2e1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:13.651112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe6873-2d84-4c65-a1b7-82e8d11bf2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:13.651549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe6873-2d84-4c65-a1b7-82e8d11bf2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:13.856880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cfbe6873-2d84-4c65-a1b7-82e8d11bf2e1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:13.858156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cfbe6873-2d84-4c65-a1b7-82e8d11bf2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:13.992539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 655/1308] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:14.333534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f467d6-e6ab-4bf4-b51a-80c7881abf86 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:14.333808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f467d6-e6ab-4bf4-b51a-80c7881abf86 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:14.334267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f467d6-e6ab-4bf4-b51a-80c7881abf86 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:14.336761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 654/1309] 10.222.0.83 () {54 vars in 965 bytes} [Wed Aug 24 11:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:14.344716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa504ee-61c6-4605-9719-9f070251b135 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:14.345356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa504ee-61c6-4605-9719-9f070251b135 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:14.345862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa504ee-61c6-4605-9719-9f070251b135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:14.459056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6aa504ee-61c6-4605-9719-9f070251b135 None None] MFA Rules not processed for user `b0145242ef784168bada8348ccd95f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:14.461276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6aa504ee-61c6-4605-9719-9f070251b135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:14.486444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:14.565774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 656/1310] 10.222.0.83 () {58 vars in 1062 bytes} [Wed Aug 24 11:16:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:14.576072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f467d6-e6ab-4bf4-b51a-80c7881abf86 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:14.578434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a3f467d6-e6ab-4bf4-b51a-80c7881abf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:14.723407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f467d6-e6ab-4bf4-b51a-80c7881abf86 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:14.829432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a800ed-e90e-4082-8002-80c18cf4b7fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:14.830346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a800ed-e90e-4082-8002-80c18cf4b7fb service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:14.830782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a800ed-e90e-4082-8002-80c18cf4b7fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:14.831223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a800ed-e90e-4082-8002-80c18cf4b7fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:14.833071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e8a800ed-e90e-4082-8002-80c18cf4b7fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:14.907813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:14.979516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a800ed-e90e-4082-8002-80c18cf4b7fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:14.982667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a800ed-e90e-4082-8002-80c18cf4b7fb service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:15.006319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 655/1311] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:17.338414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07804dca-0cbb-4102-8cba-3e28446a17a1 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:17.338891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07804dca-0cbb-4102-8cba-3e28446a17a1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:17.338983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07804dca-0cbb-4102-8cba-3e28446a17a1 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:17.340339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 657/1312] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:17.346225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb538c33-8812-4042-8e45-caa2ef492547 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:17.351749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb538c33-8812-4042-8e45-caa2ef492547 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:17.351749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb538c33-8812-4042-8e45-caa2ef492547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:17.426634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-eb538c33-8812-4042-8e45-caa2ef492547 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:17.426634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-eb538c33-8812-4042-8e45-caa2ef492547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:17.524933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 656/1313] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:17.806257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07804dca-0cbb-4102-8cba-3e28446a17a1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:17.807833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-07804dca-0cbb-4102-8cba-3e28446a17a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:17.912524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07804dca-0cbb-4102-8cba-3e28446a17a1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:17.984971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:17.987543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7e255-66bb-4510-91fe-bf14c0a7b214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:17.988111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7e255-66bb-4510-91fe-bf14c0a7b214 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:17.988389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7e255-66bb-4510-91fe-bf14c0a7b214 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:17.988659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7e255-66bb-4510-91fe-bf14c0a7b214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:17.990294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fcb7e255-66bb-4510-91fe-bf14c0a7b214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:18.105685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7e255-66bb-4510-91fe-bf14c0a7b214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:18.107075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7e255-66bb-4510-91fe-bf14c0a7b214 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:18.126627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 658/1314] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:20.578349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d34e9-c540-46a1-a774-e037b27e7348 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:20.578861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d34e9-c540-46a1-a774-e037b27e7348 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:20.578861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d34e9-c540-46a1-a774-e037b27e7348 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:20.580150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 657/1315] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:20.592726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07955d37-bedd-44d1-bf40-f350eb2c0cb2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:20.593189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07955d37-bedd-44d1-bf40-f350eb2c0cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:20.593310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07955d37-bedd-44d1-bf40-f350eb2c0cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:20.660245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-07955d37-bedd-44d1-bf40-f350eb2c0cb2 None None] MFA Rules not processed for user `e22fbb4f1849455e917f7324ffd888a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:20.661464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-07955d37-bedd-44d1-bf40-f350eb2c0cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:20.715604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:20.761463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 659/1316] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:21.050589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d34e9-c540-46a1-a774-e037b27e7348 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:21.052183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-457d34e9-c540-46a1-a774-e037b27e7348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:21.063259 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:21.158654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d34e9-c540-46a1-a774-e037b27e7348 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:21.236435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73890f7c-a60c-48e1-9681-95334291390d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:21.237042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73890f7c-a60c-48e1-9681-95334291390d service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:21.237320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73890f7c-a60c-48e1-9681-95334291390d service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:21.237588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73890f7c-a60c-48e1-9681-95334291390d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:21.239265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-73890f7c-a60c-48e1-9681-95334291390d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:21.305285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:21.353378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73890f7c-a60c-48e1-9681-95334291390d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:21.354912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73890f7c-a60c-48e1-9681-95334291390d service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:21.375172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 658/1317] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:24.133194 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec3170-a00c-4c04-9e36-1bc846d12a04 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:24.133729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec3170-a00c-4c04-9e36-1bc846d12a04 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:24.133729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec3170-a00c-4c04-9e36-1bc846d12a04 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:24.135135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 660/1318] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:24.140613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d1377-a086-49e7-b188-ac697e43472b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:24.141097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d1377-a086-49e7-b188-ac697e43472b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:24.141196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d1377-a086-49e7-b188-ac697e43472b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:24.208757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-084d1377-a086-49e7-b188-ac697e43472b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:24.210003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-084d1377-a086-49e7-b188-ac697e43472b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:24.305447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 659/1319] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:24.600441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acedd16-9726-412f-82bf-a48b098a70d4 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:24.600729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acedd16-9726-412f-82bf-a48b098a70d4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:24.601035 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acedd16-9726-412f-82bf-a48b098a70d4 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:24.602435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 661/1320] 10.222.0.83 () {54 vars in 965 bytes} [Wed Aug 24 11:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:24.608405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4fd04-c62d-410f-b7a0-c324a141c3cc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:24.608691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4fd04-c62d-410f-b7a0-c324a141c3cc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:24.608955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4fd04-c62d-410f-b7a0-c324a141c3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:24.631459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:24.688795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-66a4fd04-c62d-410f-b7a0-c324a141c3cc None None] MFA Rules not processed for user `b0145242ef784168bada8348ccd95f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:24.690078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-66a4fd04-c62d-410f-b7a0-c324a141c3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:24.765919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 660/1321] 10.222.0.83 () {58 vars in 1062 bytes} [Wed Aug 24 11:16:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:24.775768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acedd16-9726-412f-82bf-a48b098a70d4 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:24.777308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9acedd16-9726-412f-82bf-a48b098a70d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:24.887859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acedd16-9726-412f-82bf-a48b098a70d4 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:24.962113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdefb08c-e662-48df-a1a6-66818664725b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:24.962801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefb08c-e662-48df-a1a6-66818664725b service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:24.963082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefb08c-e662-48df-a1a6-66818664725b service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:24.963358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefb08c-e662-48df-a1a6-66818664725b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:24.964943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fdefb08c-e662-48df-a1a6-66818664725b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:24.972628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:25.082485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdefb08c-e662-48df-a1a6-66818664725b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:25.083940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdefb08c-e662-48df-a1a6-66818664725b service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:25.101984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 662/1322] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:28.712748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61f31c-07c7-4199-8316-b1554935ce12 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:28.713279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61f31c-07c7-4199-8316-b1554935ce12 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:28.713628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61f31c-07c7-4199-8316-b1554935ce12 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:28.716132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 661/1323] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:28.724463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4387b-e744-4dba-bd03-e62b2c162868 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:28.724967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4387b-e744-4dba-bd03-e62b2c162868 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:28.726090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4387b-e744-4dba-bd03-e62b2c162868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:28.810618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e0f4387b-e744-4dba-bd03-e62b2c162868 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:28.811824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e0f4387b-e744-4dba-bd03-e62b2c162868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:28.876315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:28.919021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 663/1324] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:29.205824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61f31c-07c7-4199-8316-b1554935ce12 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:29.207496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4d61f31c-07c7-4199-8316-b1554935ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:29.322226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61f31c-07c7-4199-8316-b1554935ce12 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:29.335919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:29.401017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648934c-ee49-4fcd-81aa-5058957e9c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:29.401630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648934c-ee49-4fcd-81aa-5058957e9c34 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:29.401945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648934c-ee49-4fcd-81aa-5058957e9c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:29.402221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648934c-ee49-4fcd-81aa-5058957e9c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:29.403878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d648934c-ee49-4fcd-81aa-5058957e9c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:29.519665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648934c-ee49-4fcd-81aa-5058957e9c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:29.521090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648934c-ee49-4fcd-81aa-5058957e9c34 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:29.540789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 662/1325] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:31.572898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c53d5-093a-4986-8ef7-4eb464342a35 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:31.573503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c53d5-093a-4986-8ef7-4eb464342a35 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:31.573503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c53d5-093a-4986-8ef7-4eb464342a35 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:31.574815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 664/1326] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:31.580754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691f31f-371d-4797-9cd4-e980310a544e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:31.581201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691f31f-371d-4797-9cd4-e980310a544e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:31.581278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691f31f-371d-4797-9cd4-e980310a544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:31.636723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:31.657613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d691f31f-371d-4797-9cd4-e980310a544e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:31.658888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d691f31f-371d-4797-9cd4-e980310a544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:31.762674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 663/1327] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:32.048063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c53d5-093a-4986-8ef7-4eb464342a35 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:32.049644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b44c53d5-093a-4986-8ef7-4eb464342a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:32.161466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c53d5-093a-4986-8ef7-4eb464342a35 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:32.247886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4aaa3-3c5c-4cf4-b198-107e5cf60604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:32.248486 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4aaa3-3c5c-4cf4-b198-107e5cf60604 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:32.248762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4aaa3-3c5c-4cf4-b198-107e5cf60604 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:32.249535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4aaa3-3c5c-4cf4-b198-107e5cf60604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:32.250678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8ec4aaa3-3c5c-4cf4-b198-107e5cf60604 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:32.270210 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:32.372645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4aaa3-3c5c-4cf4-b198-107e5cf60604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:32.374129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4aaa3-3c5c-4cf4-b198-107e5cf60604 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:32.393687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 665/1328] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:34.384899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923b047-2037-4905-bb0e-1f01cffdee2f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:34.385429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923b047-2037-4905-bb0e-1f01cffdee2f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:34.385429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923b047-2037-4905-bb0e-1f01cffdee2f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:34.386768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 664/1329] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:34.392834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c6baa-e45d-4364-84f2-7981511063fb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:34.393551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c6baa-e45d-4364-84f2-7981511063fb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:34.393551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c6baa-e45d-4364-84f2-7981511063fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:34.468639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-528c6baa-e45d-4364-84f2-7981511063fb None None] MFA Rules not processed for user `e22fbb4f1849455e917f7324ffd888a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:34.469948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-528c6baa-e45d-4364-84f2-7981511063fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:34.544468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:34.578781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 666/1330] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:34.885528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923b047-2037-4905-bb0e-1f01cffdee2f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:34.888549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7923b047-2037-4905-bb0e-1f01cffdee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:35.008544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923b047-2037-4905-bb0e-1f01cffdee2f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:35.030155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:35.088143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af5327-5795-4883-83f8-53893ef130ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:35.088750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af5327-5795-4883-83f8-53893ef130ae service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:35.089040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af5327-5795-4883-83f8-53893ef130ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:35.089291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af5327-5795-4883-83f8-53893ef130ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:35.090968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e3af5327-5795-4883-83f8-53893ef130ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:35.204489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af5327-5795-4883-83f8-53893ef130ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:35.205845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af5327-5795-4883-83f8-53893ef130ae service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:35.224756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 665/1331] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:37.591375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e605e-0ca1-4ed0-9a31-42551c10d8a2 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:37.591923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e605e-0ca1-4ed0-9a31-42551c10d8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:37.591923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e605e-0ca1-4ed0-9a31-42551c10d8a2 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:37.593436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 667/1332] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:37.599103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afce6d-e7b3-4ed0-8593-2f4b643dd3b8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:37.599540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afce6d-e7b3-4ed0-8593-2f4b643dd3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:37.603418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afce6d-e7b3-4ed0-8593-2f4b643dd3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:37.666669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:37.669512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d6afce6d-e7b3-4ed0-8593-2f4b643dd3b8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:37.670773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d6afce6d-e7b3-4ed0-8593-2f4b643dd3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:37.790527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 666/1333] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:38.075651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e605e-0ca1-4ed0-9a31-42551c10d8a2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:38.077193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9f0e605e-0ca1-4ed0-9a31-42551c10d8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:38.182256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e605e-0ca1-4ed0-9a31-42551c10d8a2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:38.257391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb6171-1448-41f4-990c-e21154f57bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:38.258020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb6171-1448-41f4-990c-e21154f57bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:38.258300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb6171-1448-41f4-990c-e21154f57bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:38.258574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb6171-1448-41f4-990c-e21154f57bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:38.260182 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c5bb6171-1448-41f4-990c-e21154f57bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:38.287704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:38.377287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb6171-1448-41f4-990c-e21154f57bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:38.378758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb6171-1448-41f4-990c-e21154f57bd0 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:38.397119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 668/1334] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:40.374158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226d6a9-4643-4a2b-8665-323e4a6de584 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:40.374767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226d6a9-4643-4a2b-8665-323e4a6de584 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:40.374767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226d6a9-4643-4a2b-8665-323e4a6de584 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:40.376088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 667/1335] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:40.382194 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d2c94-9bf9-4089-9a7c-9dde92010d44 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:40.382616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d2c94-9bf9-4089-9a7c-9dde92010d44 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:40.382749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d2c94-9bf9-4089-9a7c-9dde92010d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:40.451034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-968d2c94-9bf9-4089-9a7c-9dde92010d44 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:40.452258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-968d2c94-9bf9-4089-9a7c-9dde92010d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:40.531733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:40.555384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 669/1336] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:40.850886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226d6a9-4643-4a2b-8665-323e4a6de584 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:40.852372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a226d6a9-4643-4a2b-8665-323e4a6de584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:40.958470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226d6a9-4643-4a2b-8665-323e4a6de584 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:40.987842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:41.034303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657908cb-51ee-4bf2-ba54-c7bc0fb243d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:41.034923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657908cb-51ee-4bf2-ba54-c7bc0fb243d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:41.035199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657908cb-51ee-4bf2-ba54-c7bc0fb243d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:41.035471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657908cb-51ee-4bf2-ba54-c7bc0fb243d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:41.037146 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-657908cb-51ee-4bf2-ba54-c7bc0fb243d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:41.147902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657908cb-51ee-4bf2-ba54-c7bc0fb243d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:41.149266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657908cb-51ee-4bf2-ba54-c7bc0fb243d6 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:41.168881 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 668/1337] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:43.671425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb8bca-4605-4635-803f-e21fe387d036 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:43.672282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb8bca-4605-4635-803f-e21fe387d036 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:43.672282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb8bca-4605-4635-803f-e21fe387d036 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:43.673613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 670/1338] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:43.679787 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2fbbf-976c-4340-8e16-6df3907f9b5e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:43.680343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2fbbf-976c-4340-8e16-6df3907f9b5e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:43.680343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2fbbf-976c-4340-8e16-6df3907f9b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:43.748160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b3a2fbbf-976c-4340-8e16-6df3907f9b5e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:43.749387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b3a2fbbf-976c-4340-8e16-6df3907f9b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:43.758722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:43.852344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 669/1339] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:16:44.140195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb8bca-4605-4635-803f-e21fe387d036 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:44.141728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-96fb8bca-4605-4635-803f-e21fe387d036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:44.250329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb8bca-4605-4635-803f-e21fe387d036 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:44.326430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b540c-810c-40fd-b8b0-2c7bf0d92948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:44.327047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b540c-810c-40fd-b8b0-2c7bf0d92948 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:44.327326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b540c-810c-40fd-b8b0-2c7bf0d92948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:44.327598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b540c-810c-40fd-b8b0-2c7bf0d92948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:44.329266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-761b540c-810c-40fd-b8b0-2c7bf0d92948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:44.368708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:44.446181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b540c-810c-40fd-b8b0-2c7bf0d92948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:44.447656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b540c-810c-40fd-b8b0-2c7bf0d92948 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:44.466183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 671/1340] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:48.945142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780d91-d452-4760-b1f8-6707dd45db14 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:48.945782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780d91-d452-4760-b1f8-6707dd45db14 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:48.945782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780d91-d452-4760-b1f8-6707dd45db14 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:48.947782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 670/1341] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:48.953623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014572d-9a7d-4bd0-942f-5aabb042910b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:48.954297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014572d-9a7d-4bd0-942f-5aabb042910b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:48.954297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014572d-9a7d-4bd0-942f-5aabb042910b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.031114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2014572d-9a7d-4bd0-942f-5aabb042910b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:49.032679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2014572d-9a7d-4bd0-942f-5aabb042910b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:49.130876 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:49.142535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 672/1342] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:49.431289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780d91-d452-4760-b1f8-6707dd45db14 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:49.432964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ee780d91-d452-4760-b1f8-6707dd45db14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:49.557561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780d91-d452-4760-b1f8-6707dd45db14 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:49.606864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:49.649363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f3d20-65b3-4d8e-88d1-146bfdcc69c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:49.649850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3d20-65b3-4d8e-88d1-146bfdcc69c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:49.650156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3d20-65b3-4d8e-88d1-146bfdcc69c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:49.650432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3d20-65b3-4d8e-88d1-146bfdcc69c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:49.652183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0f8f3d20-65b3-4d8e-88d1-146bfdcc69c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:49.774752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f3d20-65b3-4d8e-88d1-146bfdcc69c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:49.776215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f3d20-65b3-4d8e-88d1-146bfdcc69c5 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:49.796133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 671/1343] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:52.878105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49428fdb-ce7d-4817-b134-152bdab59366 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:52.878646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49428fdb-ce7d-4817-b134-152bdab59366 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:52.878646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49428fdb-ce7d-4817-b134-152bdab59366 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:52.879932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 673/1344] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:52.887820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e5c20-19c2-40d9-8ceb-7ea0a81f7183 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:52.888406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e5c20-19c2-40d9-8ceb-7ea0a81f7183 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:52.888406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e5c20-19c2-40d9-8ceb-7ea0a81f7183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:52.982955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bb3e5c20-19c2-40d9-8ceb-7ea0a81f7183 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:52.984188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bb3e5c20-19c2-40d9-8ceb-7ea0a81f7183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:53.003133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:53.088639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 672/1345] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:53.380113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49428fdb-ce7d-4817-b134-152bdab59366 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:53.381659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-49428fdb-ce7d-4817-b134-152bdab59366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:53.494975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49428fdb-ce7d-4817-b134-152bdab59366 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:53.575541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b3a07-0484-4575-8f71-cfb138851b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:53.576155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3a07-0484-4575-8f71-cfb138851b67 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:53.576438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3a07-0484-4575-8f71-cfb138851b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:53.576714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3a07-0484-4575-8f71-cfb138851b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:53.578472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d97b3a07-0484-4575-8f71-cfb138851b67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:53.643047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:53.712041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b3a07-0484-4575-8f71-cfb138851b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:53.713422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b3a07-0484-4575-8f71-cfb138851b67 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:53.732788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 674/1346] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:16:58.197183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0268083-329e-4ce1-b279-cdee534e2e73 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:58.197709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0268083-329e-4ce1-b279-cdee534e2e73 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:58.197709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0268083-329e-4ce1-b279-cdee534e2e73 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:58.199125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 673/1347] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:16:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:16:58.205074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dec79c-210f-417c-9427-2cb279aa9ec7 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:58.205564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dec79c-210f-417c-9427-2cb279aa9ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:58.205564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dec79c-210f-417c-9427-2cb279aa9ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:58.283948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-95dec79c-210f-417c-9427-2cb279aa9ec7 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:16:58.285230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-95dec79c-210f-417c-9427-2cb279aa9ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:58.389446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:58.392155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 675/1348] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:16:58.675671 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0268083-329e-4ce1-b279-cdee534e2e73 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:16:58.677218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a0268083-329e-4ce1-b279-cdee534e2e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:58.790845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0268083-329e-4ce1-b279-cdee534e2e73 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:16:58.839939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:58.867324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681951a3-c9aa-4f76-9073-7ee14c49fa49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:16:58.868526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681951a3-c9aa-4f76-9073-7ee14c49fa49 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:16:58.869011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681951a3-c9aa-4f76-9073-7ee14c49fa49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:16:58.869457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681951a3-c9aa-4f76-9073-7ee14c49fa49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:16:58.871445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-681951a3-c9aa-4f76-9073-7ee14c49fa49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:16:58.991587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681951a3-c9aa-4f76-9073-7ee14c49fa49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:16:58.993630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681951a3-c9aa-4f76-9073-7ee14c49fa49 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:16:59.013374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 674/1349] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:04.063501 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ece3b6-1a56-4f75-9f3e-046ca53646c6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:04.063984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ece3b6-1a56-4f75-9f3e-046ca53646c6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:04.064065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ece3b6-1a56-4f75-9f3e-046ca53646c6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:04.065710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 676/1350] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:04.071305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121a223-a376-47a6-8600-d574161f5e8b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:04.072296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121a223-a376-47a6-8600-d574161f5e8b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:04.072576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121a223-a376-47a6-8600-d574161f5e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:04.203894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3121a223-a376-47a6-8600-d574161f5e8b None None] MFA Rules not processed for user `eb76ad667a744126ab00b38c346e9017`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:04.205347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3121a223-a376-47a6-8600-d574161f5e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:04.241324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:04.394722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 675/1351] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:17:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:17:08.634219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77ef0-6979-494a-b042-94c69976d184 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:08.634750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77ef0-6979-494a-b042-94c69976d184 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:08.634750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77ef0-6979-494a-b042-94c69976d184 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:08.636236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 677/1352] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:17:08.641212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60779b-d5a5-4e72-883b-e95861229d62 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:08.641801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60779b-d5a5-4e72-883b-e95861229d62 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:08.641801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60779b-d5a5-4e72-883b-e95861229d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:08.731262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ca60779b-d5a5-4e72-883b-e95861229d62 None None] MFA Rules not processed for user `eb76ad667a744126ab00b38c346e9017`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:08.732524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ca60779b-d5a5-4e72-883b-e95861229d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:08.821590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:08.882031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 676/1353] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:17:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:10.310229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb88804-8a1e-49f6-9d29-72e8d77a7275 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:10.310805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb88804-8a1e-49f6-9d29-72e8d77a7275 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:10.310805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb88804-8a1e-49f6-9d29-72e8d77a7275 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:10.312129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 678/1354] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:10.317369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f0149-a5b9-4332-aafe-cd1b47aed3f0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:10.317803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f0149-a5b9-4332-aafe-cd1b47aed3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:10.318027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f0149-a5b9-4332-aafe-cd1b47aed3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:10.402737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-470f0149-a5b9-4332-aafe-cd1b47aed3f0 None None] MFA Rules not processed for user `eb76ad667a744126ab00b38c346e9017`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:10.403953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-470f0149-a5b9-4332-aafe-cd1b47aed3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:10.517836 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 677/1355] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:17:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:14.002331 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779ba41-b3ac-401c-b4bf-971bce45449b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:14.002897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779ba41-b3ac-401c-b4bf-971bce45449b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:14.002897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779ba41-b3ac-401c-b4bf-971bce45449b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:14.004131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 679/1356] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:14.009998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47072a02-0b13-4895-a665-7bf7f06c70a9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:14.010742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47072a02-0b13-4895-a665-7bf7f06c70a9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:14.010742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47072a02-0b13-4895-a665-7bf7f06c70a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:14.059217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:14.094371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-47072a02-0b13-4895-a665-7bf7f06c70a9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:14.095749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-47072a02-0b13-4895-a665-7bf7f06c70a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:14.218360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 678/1357] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:14.535082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e45d25-b7e6-4158-80cc-6b2fa0029cd6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:14.535584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e45d25-b7e6-4158-80cc-6b2fa0029cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:14.536141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e45d25-b7e6-4158-80cc-6b2fa0029cd6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:14.538098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 680/1358] 10.222.0.83 () {54 vars in 962 bytes} [Wed Aug 24 11:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:14.547436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fd0bc-693e-444c-9aed-00932ee71ce7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:14.548088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fd0bc-693e-444c-9aed-00932ee71ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:14.548527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fd0bc-693e-444c-9aed-00932ee71ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:14.652851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-335fd0bc-693e-444c-9aed-00932ee71ce7 None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:14.654123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-335fd0bc-693e-444c-9aed-00932ee71ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:14.718670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:14.793455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 679/1359] 10.222.0.83 () {58 vars in 1059 bytes} [Wed Aug 24 11:17:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:14.802211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e45d25-b7e6-4158-80cc-6b2fa0029cd6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:14.804273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-39e45d25-b7e6-4158-80cc-6b2fa0029cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:14.967810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e45d25-b7e6-4158-80cc-6b2fa0029cd6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:15.052045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:15.105420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4825fde-623a-40f9-a108-cf649039fa12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:15.106053 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4825fde-623a-40f9-a108-cf649039fa12 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:15.106336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4825fde-623a-40f9-a108-cf649039fa12 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:15.106599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4825fde-623a-40f9-a108-cf649039fa12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:15.108560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f4825fde-623a-40f9-a108-cf649039fa12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:15.210982 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4825fde-623a-40f9-a108-cf649039fa12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:15.212654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4825fde-623a-40f9-a108-cf649039fa12 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:15.232915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 681/1360] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:17.437062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06f7aa-3c79-46a6-8efa-ff50482b5a05 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:17.437612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06f7aa-3c79-46a6-8efa-ff50482b5a05 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:17.437612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06f7aa-3c79-46a6-8efa-ff50482b5a05 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:17.439220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 680/1361] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:17.444681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0576c-fe91-4c64-8fbc-4ad360786cb6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:17.445237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0576c-fe91-4c64-8fbc-4ad360786cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:17.445237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0576c-fe91-4c64-8fbc-4ad360786cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:17.516245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-20d0576c-fe91-4c64-8fbc-4ad360786cb6 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:17.517491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-20d0576c-fe91-4c64-8fbc-4ad360786cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:17.559596 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:17.634758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 682/1362] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:17.933267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06f7aa-3c79-46a6-8efa-ff50482b5a05 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:17.935072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bc06f7aa-3c79-46a6-8efa-ff50482b5a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:18.126670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06f7aa-3c79-46a6-8efa-ff50482b5a05 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:18.137342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:18.269763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818bfb2-1c85-431e-b8a8-3d6a6a1312d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:18.270420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818bfb2-1c85-431e-b8a8-3d6a6a1312d3 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:18.270716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818bfb2-1c85-431e-b8a8-3d6a6a1312d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:18.270970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818bfb2-1c85-431e-b8a8-3d6a6a1312d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:18.272609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e818bfb2-1c85-431e-b8a8-3d6a6a1312d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:18.381329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818bfb2-1c85-431e-b8a8-3d6a6a1312d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:18.383117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818bfb2-1c85-431e-b8a8-3d6a6a1312d3 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:18.408443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 681/1363] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:20.816789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165dd55-9f8b-4463-ba47-7f0f96928a4f None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:20.817297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165dd55-9f8b-4463-ba47-7f0f96928a4f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:20.817404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165dd55-9f8b-4463-ba47-7f0f96928a4f None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:20.819425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 683/1364] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:20.825246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fb89c-73e3-4280-92f0-80dce11f6b65 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:20.826318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fb89c-73e3-4280-92f0-80dce11f6b65 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:20.827924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fb89c-73e3-4280-92f0-80dce11f6b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:20.846211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:20.905692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-489fb89c-73e3-4280-92f0-80dce11f6b65 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:20.907089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-489fb89c-73e3-4280-92f0-80dce11f6b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:21.019318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 682/1365] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:21.321159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1abfd-5467-45f0-a23e-332c28dc4b92 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:21.321444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1abfd-5467-45f0-a23e-332c28dc4b92 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:21.321921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1abfd-5467-45f0-a23e-332c28dc4b92 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:21.323767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 684/1366] 10.222.0.83 () {54 vars in 962 bytes} [Wed Aug 24 11:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:21.334361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f3a6c-c003-4e19-b33f-eb395f2570aa None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:21.334658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f3a6c-c003-4e19-b33f-eb395f2570aa None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:21.335015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f3a6c-c003-4e19-b33f-eb395f2570aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:21.423746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c05f3a6c-c003-4e19-b33f-eb395f2570aa None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:21.424817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c05f3a6c-c003-4e19-b33f-eb395f2570aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:21.465618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:21.557347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 683/1367] 10.222.0.83 () {58 vars in 1059 bytes} [Wed Aug 24 11:17:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:21.565443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1abfd-5467-45f0-a23e-332c28dc4b92 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:21.567007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-37e1abfd-5467-45f0-a23e-332c28dc4b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:21.736046 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1abfd-5467-45f0-a23e-332c28dc4b92 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:21.760216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:21.864180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d7150-f6cd-42db-9a59-25df1a24e938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:21.865091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d7150-f6cd-42db-9a59-25df1a24e938 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:21.865091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d7150-f6cd-42db-9a59-25df1a24e938 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:21.865280 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d7150-f6cd-42db-9a59-25df1a24e938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:21.867038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2c8d7150-f6cd-42db-9a59-25df1a24e938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:21.964908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d7150-f6cd-42db-9a59-25df1a24e938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:21.966709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d7150-f6cd-42db-9a59-25df1a24e938 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:21.987970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 685/1368] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:22.643767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce49f1-a769-44d1-82ef-7792ea6200ec None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:22.644349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce49f1-a769-44d1-82ef-7792ea6200ec None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:22.644349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce49f1-a769-44d1-82ef-7792ea6200ec None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:22.645984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 684/1369] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:22.655823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeab43-67ff-48eb-9109-ea0238ad319a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:22.656228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeab43-67ff-48eb-9109-ea0238ad319a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:22.657060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeab43-67ff-48eb-9109-ea0238ad319a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:22.700102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:22.754174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-faeeab43-67ff-48eb-9109-ea0238ad319a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:22.755597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-faeeab43-67ff-48eb-9109-ea0238ad319a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:22.906320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 686/1370] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:23.507303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a21da-69a4-42ec-a6fb-e17356494839 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:23.508643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a21da-69a4-42ec-a6fb-e17356494839 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:23.508643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a21da-69a4-42ec-a6fb-e17356494839 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:23.510031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 685/1371] 10.222.0.83 () {54 vars in 964 bytes} [Wed Aug 24 11:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:23.515287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ffd30-c849-4042-be42-8bf615cca125 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:23.515578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ffd30-c849-4042-be42-8bf615cca125 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:23.515857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ffd30-c849-4042-be42-8bf615cca125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:23.622029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ff3ffd30-c849-4042-be42-8bf615cca125 None None] MFA Rules not processed for user `f9dd6f187dd84f04866da64732b242d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:23.624341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ff3ffd30-c849-4042-be42-8bf615cca125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:23.681504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:23.725172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 687/1372] 10.222.0.83 () {58 vars in 1061 bytes} [Wed Aug 24 11:17:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:23.725803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a21da-69a4-42ec-a6fb-e17356494839 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:23.727946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7c9a21da-69a4-42ec-a6fb-e17356494839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:23.857994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a21da-69a4-42ec-a6fb-e17356494839 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:23.908206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:23.956140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8879c8-f53f-4dff-b096-376617c342d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:23.956767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8879c8-f53f-4dff-b096-376617c342d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:23.957049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8879c8-f53f-4dff-b096-376617c342d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:23.957463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8879c8-f53f-4dff-b096-376617c342d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:23.959121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bf8879c8-f53f-4dff-b096-376617c342d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:24.090142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8879c8-f53f-4dff-b096-376617c342d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:24.091504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8879c8-f53f-4dff-b096-376617c342d0 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:24.118908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 686/1373] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:24.206762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b2eba-e63f-485f-a967-ae16e1509c09 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:24.207212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b2eba-e63f-485f-a967-ae16e1509c09 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:24.207399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b2eba-e63f-485f-a967-ae16e1509c09 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:24.208918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 688/1374] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:24.214159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4ca5c-ac85-4a73-b202-ce22af1380d6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:24.214519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4ca5c-ac85-4a73-b202-ce22af1380d6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:24.214704 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4ca5c-ac85-4a73-b202-ce22af1380d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:24.309674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c7f4ca5c-ac85-4a73-b202-ce22af1380d6 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:24.310936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c7f4ca5c-ac85-4a73-b202-ce22af1380d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:24.335939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:24.418395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 687/1375] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:17:24.714908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b2eba-e63f-485f-a967-ae16e1509c09 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:24.716687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-247b2eba-e63f-485f-a967-ae16e1509c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:24.872269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b2eba-e63f-485f-a967-ae16e1509c09 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:24.928302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:24.992976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be61ce-260c-42e1-8238-445e37caaede service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:24.993488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be61ce-260c-42e1-8238-445e37caaede service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:24.993759 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be61ce-260c-42e1-8238-445e37caaede service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:24.994058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be61ce-260c-42e1-8238-445e37caaede service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:24.995658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d0be61ce-260c-42e1-8238-445e37caaede service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:25.094583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be61ce-260c-42e1-8238-445e37caaede service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:25.096193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be61ce-260c-42e1-8238-445e37caaede service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:25.116258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 689/1376] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:27.244465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa309b3-0cdc-4782-b98f-25e64be7b61e None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:27.244988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa309b3-0cdc-4782-b98f-25e64be7b61e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:27.245076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa309b3-0cdc-4782-b98f-25e64be7b61e None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:27.246461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 688/1377] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:27.252359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7ebd-e407-479a-81b5-2209a288ab29 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:27.252830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7ebd-e407-479a-81b5-2209a288ab29 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:27.253038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7ebd-e407-479a-81b5-2209a288ab29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:27.338580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-de9b7ebd-e407-479a-81b5-2209a288ab29 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:27.339863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-de9b7ebd-e407-479a-81b5-2209a288ab29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:27.346896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:27.458797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 690/1378] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:27.756425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa309b3-0cdc-4782-b98f-25e64be7b61e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:27.758222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7aa309b3-0cdc-4782-b98f-25e64be7b61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:27.973916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa309b3-0cdc-4782-b98f-25e64be7b61e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:27.979055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:28.115482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcee605-b819-4d52-b63d-5823d1d82c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:28.116230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcee605-b819-4d52-b63d-5823d1d82c1c service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:28.116574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcee605-b819-4d52-b63d-5823d1d82c1c service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:28.116875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcee605-b819-4d52-b63d-5823d1d82c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:28.118445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fdcee605-b819-4d52-b63d-5823d1d82c1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:28.230895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcee605-b819-4d52-b63d-5823d1d82c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:28.232558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcee605-b819-4d52-b63d-5823d1d82c1c service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:28.255984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 689/1379] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:30.433427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8373904-399e-496d-86ea-faedde11062a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:30.434081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8373904-399e-496d-86ea-faedde11062a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:30.434081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8373904-399e-496d-86ea-faedde11062a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:30.435538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 691/1380] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:30.440859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342c192-8fc4-4ed1-99cd-54d53336893a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:30.441343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342c192-8fc4-4ed1-99cd-54d53336893a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:30.441446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342c192-8fc4-4ed1-99cd-54d53336893a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:30.454326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:30.517363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2342c192-8fc4-4ed1-99cd-54d53336893a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:30.518620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2342c192-8fc4-4ed1-99cd-54d53336893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:30.625930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 690/1381] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:17:30.930418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8373904-399e-496d-86ea-faedde11062a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:30.931920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e8373904-399e-496d-86ea-faedde11062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:31.089684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:31.102389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8373904-399e-496d-86ea-faedde11062a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:31.232153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe756c8-f13d-4b5a-ac29-4fd228d7487d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:31.232953 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe756c8-f13d-4b5a-ac29-4fd228d7487d service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:31.233302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe756c8-f13d-4b5a-ac29-4fd228d7487d service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:31.233652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe756c8-f13d-4b5a-ac29-4fd228d7487d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:31.235388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6fe756c8-f13d-4b5a-ac29-4fd228d7487d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:31.350755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe756c8-f13d-4b5a-ac29-4fd228d7487d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:31.352488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe756c8-f13d-4b5a-ac29-4fd228d7487d service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:31.365579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:31.375230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 692/1382] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:33.579490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af103ce4-9de9-4c2d-aa8f-714bfc9855f0 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:33.580012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af103ce4-9de9-4c2d-aa8f-714bfc9855f0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:33.580012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af103ce4-9de9-4c2d-aa8f-714bfc9855f0 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:33.581375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 691/1383] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:33.587192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70502-3cce-406d-b9ab-562d01395571 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:33.588453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70502-3cce-406d-b9ab-562d01395571 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:33.588743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70502-3cce-406d-b9ab-562d01395571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:33.671776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d9e70502-3cce-406d-b9ab-562d01395571 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:33.673014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d9e70502-3cce-406d-b9ab-562d01395571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:33.786790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 693/1384] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:34.151793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af103ce4-9de9-4c2d-aa8f-714bfc9855f0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:34.153344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-af103ce4-9de9-4c2d-aa8f-714bfc9855f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:34.257828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:34.329086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af103ce4-9de9-4c2d-aa8f-714bfc9855f0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:34.493295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aef199-afec-4227-af74-143446e263d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:34.493928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aef199-afec-4227-af74-143446e263d9 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:34.494284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aef199-afec-4227-af74-143446e263d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:34.494631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aef199-afec-4227-af74-143446e263d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:34.496307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d1aef199-afec-4227-af74-143446e263d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:34.554858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:34.606760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aef199-afec-4227-af74-143446e263d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:34.608820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aef199-afec-4227-af74-143446e263d9 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:34.631122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 692/1385] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:36.596191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970d689-0250-4b8b-8d55-b7012094b459 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:36.596742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970d689-0250-4b8b-8d55-b7012094b459 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:36.596742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970d689-0250-4b8b-8d55-b7012094b459 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:36.598335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 694/1386] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:36.604007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412debfc-bca5-4bd1-9049-6ed1c72c3a3e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:36.604473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412debfc-bca5-4bd1-9049-6ed1c72c3a3e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:36.604552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412debfc-bca5-4bd1-9049-6ed1c72c3a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:36.677100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-412debfc-bca5-4bd1-9049-6ed1c72c3a3e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:36.678340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-412debfc-bca5-4bd1-9049-6ed1c72c3a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:36.794943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 693/1387] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:37.092446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970d689-0250-4b8b-8d55-b7012094b459 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:37.094033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c970d689-0250-4b8b-8d55-b7012094b459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:37.220315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:37.308462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970d689-0250-4b8b-8d55-b7012094b459 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:37.462107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cc15b-bf06-44b5-8ef9-ffd80950c949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:37.462713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cc15b-bf06-44b5-8ef9-ffd80950c949 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:37.462989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cc15b-bf06-44b5-8ef9-ffd80950c949 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:37.463258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cc15b-bf06-44b5-8ef9-ffd80950c949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:37.464880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-147cc15b-bf06-44b5-8ef9-ffd80950c949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:37.506217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:37.563610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cc15b-bf06-44b5-8ef9-ffd80950c949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:37.565228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cc15b-bf06-44b5-8ef9-ffd80950c949 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:37.584570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 695/1388] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:17:39.654380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3497e-20d7-40d4-827a-82ef37b3f843 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:39.654970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3497e-20d7-40d4-827a-82ef37b3f843 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:39.654970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3497e-20d7-40d4-827a-82ef37b3f843 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:39.656220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 694/1389] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:39.661760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc322-92b9-401f-81fb-512982514ebc None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:39.662307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc322-92b9-401f-81fb-512982514ebc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:39.662406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc322-92b9-401f-81fb-512982514ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:39.685629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d9705-5b6d-4887-a528-f8431659a128 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:39.686266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d9705-5b6d-4887-a528-f8431659a128 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:39.686266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d9705-5b6d-4887-a528-f8431659a128 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:39.687539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 695/1390] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:17:39.692896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d9111-6b56-4f02-8238-26692dd1e7af None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:39.693171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d9111-6b56-4f02-8238-26692dd1e7af None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:39.693446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d9111-6b56-4f02-8238-26692dd1e7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:39.736818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:39.745849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d8fcc322-92b9-401f-81fb-512982514ebc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:39.756340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d8fcc322-92b9-401f-81fb-512982514ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:39.771804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-524d9111-6b56-4f02-8238-26692dd1e7af None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:39.773554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-524d9111-6b56-4f02-8238-26692dd1e7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:39.854939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 696/1391] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:39.880018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 696/1392] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:17:40.178295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcc322-92b9-401f-81fb-512982514ebc None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:40.179062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3b85c-8610-48a1-bdbe-71689144f8ef None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:40.179277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3b85c-8610-48a1-bdbe-71689144f8ef None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:40.179553 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3b85c-8610-48a1-bdbe-71689144f8ef None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:40.179661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d8fcc322-92b9-401f-81fb-512982514ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:40.180903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 697/1393] 10.222.0.83 () {54 vars in 967 bytes} [Wed Aug 24 11:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:40.185802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09417e-9df0-4f1a-99f0-c9196fd727f0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:40.186123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09417e-9df0-4f1a-99f0-c9196fd727f0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:40.186397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09417e-9df0-4f1a-99f0-c9196fd727f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:40.194373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:40.267309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3c09417e-9df0-4f1a-99f0-c9196fd727f0 None None] MFA Rules not processed for user `f1f214c2b77d4cbab2f15122ba21bf38`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:40.268452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3c09417e-9df0-4f1a-99f0-c9196fd727f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:40.304394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcc322-92b9-401f-81fb-512982514ebc None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:40.322336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:40.352672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 698/1394] 10.222.0.83 () {58 vars in 1064 bytes} [Wed Aug 24 11:17:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:17:40.362134 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09417e-9df0-4f1a-99f0-c9196fd727f0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:40.368885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09417e-9df0-4f1a-99f0-c9196fd727f0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:40.436079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b87b78-26c7-4d50-96bf-820ad2f1a7e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:40.436679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87b78-26c7-4d50-96bf-820ad2f1a7e6 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:40.436955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87b78-26c7-4d50-96bf-820ad2f1a7e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:40.437228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87b78-26c7-4d50-96bf-820ad2f1a7e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:40.438880 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-87b87b78-26c7-4d50-96bf-820ad2f1a7e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:40.446143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfdd0c-65f2-4d1c-acd5-71bc00d329fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f214c2b77d4cbab2f15122ba21bf38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:40.446749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfdd0c-65f2-4d1c-acd5-71bc00d329fe service glance] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:40.447120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfdd0c-65f2-4d1c-acd5-71bc00d329fe service glance] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:40.447292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfdd0c-65f2-4d1c-acd5-71bc00d329fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:40.448541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:40.448901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e6bfdd0c-65f2-4d1c-acd5-71bc00d329fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:40.543261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b87b78-26c7-4d50-96bf-820ad2f1a7e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:40.544891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b87b78-26c7-4d50-96bf-820ad2f1a7e6 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:40.565187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 697/1395] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:17:40.575821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfdd0c-65f2-4d1c-acd5-71bc00d329fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:40.576126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfdd0c-65f2-4d1c-acd5-71bc00d329fe service glance] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:40.590961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:40.595761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 699/1396] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:17:41.599561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ec331-8b9e-423d-9153-a1eebd4db6c9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:41.600081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ec331-8b9e-423d-9153-a1eebd4db6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:41.600176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ec331-8b9e-423d-9153-a1eebd4db6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:41.671885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9b8ec331-8b9e-423d-9153-a1eebd4db6c9 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:41.673495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9b8ec331-8b9e-423d-9153-a1eebd4db6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:41.763259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:41.766744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 698/1397] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:17:41.814659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d3b90-8fe4-4ddc-9e07-1633d784613a None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:41.815191 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d3b90-8fe4-4ddc-9e07-1633d784613a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:41.815191 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d3b90-8fe4-4ddc-9e07-1633d784613a None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:41.816618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 700/1398] 10.222.0.83 () {54 vars in 943 bytes} [Wed Aug 24 11:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:41.823772 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0bd0c-c929-4f2e-8a13-6638ff4f55a4 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:41.824085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0bd0c-c929-4f2e-8a13-6638ff4f55a4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:41.824362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0bd0c-c929-4f2e-8a13-6638ff4f55a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:41.906649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-abd0bd0c-c929-4f2e-8a13-6638ff4f55a4 None None] MFA Rules not processed for user `12eafcbf5c98425782138d5363db9a25`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:41.907891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-abd0bd0c-c929-4f2e-8a13-6638ff4f55a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:42.018401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 699/1399] 10.222.0.83 () {58 vars in 1040 bytes} [Wed Aug 24 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:42.027777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d3b90-8fe4-4ddc-9e07-1633d784613a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:42.029298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fb6d3b90-8fe4-4ddc-9e07-1633d784613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:42.173340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d3b90-8fe4-4ddc-9e07-1633d784613a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:42.264804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:42.286211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3094d-58a5-45ee-8d32-27edda66723e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:42.286831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3094d-58a5-45ee-8d32-27edda66723e service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:42.287118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3094d-58a5-45ee-8d32-27edda66723e service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:42.287396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3094d-58a5-45ee-8d32-27edda66723e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:42.289725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-26c3094d-58a5-45ee-8d32-27edda66723e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:42.379868 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3094d-58a5-45ee-8d32-27edda66723e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:42.381521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3094d-58a5-45ee-8d32-27edda66723e service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:42.390535 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 701/1400] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:17:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:17:42.698427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cd940-2a9a-4769-a81e-49f8a838835c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:42.699032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cd940-2a9a-4769-a81e-49f8a838835c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:42.699032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cd940-2a9a-4769-a81e-49f8a838835c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:42.700307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 700/1401] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:42.705806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b524034-6fde-4f7e-a343-27c4903d1fe4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:42.706127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b524034-6fde-4f7e-a343-27c4903d1fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:42.706432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b524034-6fde-4f7e-a343-27c4903d1fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:42.781455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3b524034-6fde-4f7e-a343-27c4903d1fe4 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:42.782800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3b524034-6fde-4f7e-a343-27c4903d1fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:42.854580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:42.886584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 702/1402] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:17:43.177583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cd940-2a9a-4769-a81e-49f8a838835c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:43.179740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6d9cd940-2a9a-4769-a81e-49f8a838835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:43.254413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:43.372628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cd940-2a9a-4769-a81e-49f8a838835c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:43.516387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26fe7c-0ace-4d9b-8313-e0358782484d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:43.516993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26fe7c-0ace-4d9b-8313-e0358782484d service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:43.517272 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26fe7c-0ace-4d9b-8313-e0358782484d service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:43.517546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26fe7c-0ace-4d9b-8313-e0358782484d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:43.519206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ff26fe7c-0ace-4d9b-8313-e0358782484d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:43.544079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:43.635064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26fe7c-0ace-4d9b-8313-e0358782484d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:43.636729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26fe7c-0ace-4d9b-8313-e0358782484d service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:43.660851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 701/1403] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:45.959530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ec953-5eb9-4f9d-91ea-5888748ce65a None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:45.960051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ec953-5eb9-4f9d-91ea-5888748ce65a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:45.960275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ec953-5eb9-4f9d-91ea-5888748ce65a None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:45.963132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 703/1404] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:45.967311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbdbfb-f387-43ee-9261-c374a106e7c2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:45.967750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbdbfb-f387-43ee-9261-c374a106e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:45.967841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbdbfb-f387-43ee-9261-c374a106e7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:46.044069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-15cbdbfb-f387-43ee-9261-c374a106e7c2 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:46.045295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-15cbdbfb-f387-43ee-9261-c374a106e7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:46.160363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:46.168220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 702/1405] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:46.462623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ec953-5eb9-4f9d-91ea-5888748ce65a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:46.464311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b42ec953-5eb9-4f9d-91ea-5888748ce65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:46.630973 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ec953-5eb9-4f9d-91ea-5888748ce65a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:46.701546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:46.768474 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74892c-3b52-49f2-9e3a-1ea2f5bd74e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:46.769403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74892c-3b52-49f2-9e3a-1ea2f5bd74e7 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:46.769679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74892c-3b52-49f2-9e3a-1ea2f5bd74e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:46.769979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74892c-3b52-49f2-9e3a-1ea2f5bd74e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:46.771601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bc74892c-3b52-49f2-9e3a-1ea2f5bd74e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:46.873632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74892c-3b52-49f2-9e3a-1ea2f5bd74e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:46.875446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74892c-3b52-49f2-9e3a-1ea2f5bd74e7 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:46.908103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 704/1406] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:17:49.169971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d91830-27d3-4794-a831-e21e136770af None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:49.171558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d91830-27d3-4794-a831-e21e136770af None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:49.172207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d91830-27d3-4794-a831-e21e136770af None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:49.174151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 703/1407] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:17:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:17:49.183325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fe56b-badb-4ece-b0b6-6eba0c4c710f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:49.183325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fe56b-badb-4ece-b0b6-6eba0c4c710f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:49.183325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fe56b-badb-4ece-b0b6-6eba0c4c710f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:49.296728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1e0fe56b-badb-4ece-b0b6-6eba0c4c710f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:17:49.298789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1e0fe56b-badb-4ece-b0b6-6eba0c4c710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:49.323220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:49.463065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 705/1408] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:17:49.825701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d91830-27d3-4794-a831-e21e136770af None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:17:49.827597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a3d91830-27d3-4794-a831-e21e136770af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:50.020883 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d91830-27d3-4794-a831-e21e136770af None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:17:50.115571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:17:50.168683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785fe95-769b-42eb-8989-1450453fbd46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:17:50.169288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785fe95-769b-42eb-8989-1450453fbd46 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:17:50.169568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785fe95-769b-42eb-8989-1450453fbd46 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:17:50.169839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785fe95-769b-42eb-8989-1450453fbd46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:17:50.172710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6785fe95-769b-42eb-8989-1450453fbd46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:17:50.276378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785fe95-769b-42eb-8989-1450453fbd46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:17:50.278800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785fe95-769b-42eb-8989-1450453fbd46 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:17:50.302133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 704/1409] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:17:59.444720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=38 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:00.299375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=64 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:06.886068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cc5fe-6b2f-4133-af1a-ec551e24e626 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:06.886653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cc5fe-6b2f-4133-af1a-ec551e24e626 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:06.886653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cc5fe-6b2f-4133-af1a-ec551e24e626 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:06.888000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 706/1410] 10.222.0.83 () {54 vars in 918 bytes} [Wed Aug 24 11:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:06.893602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b92c11-5e7d-4891-9f9d-d4b99a5a1ea7 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:06.894126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b92c11-5e7d-4891-9f9d-d4b99a5a1ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:06.894218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b92c11-5e7d-4891-9f9d-d4b99a5a1ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:06.974170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f5b92c11-5e7d-4891-9f9d-d4b99a5a1ea7 None None] MFA Rules not processed for user `eb76ad667a744126ab00b38c346e9017`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:06.975405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f5b92c11-5e7d-4891-9f9d-d4b99a5a1ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:07.085483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 705/1411] 10.222.0.83 () {58 vars in 1015 bytes} [Wed Aug 24 11:18:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:07.100161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988ab28-8d5a-4dcd-8040-41ee1aa7e2ac None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:07.100448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988ab28-8d5a-4dcd-8040-41ee1aa7e2ac None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:07.100724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988ab28-8d5a-4dcd-8040-41ee1aa7e2ac None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:07.102121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 707/1412] 10.222.0.83 () {54 vars in 918 bytes} [Wed Aug 24 11:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:07.107523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e64d6c-3657-4131-a53b-3c0b0d017641 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:07.107808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e64d6c-3657-4131-a53b-3c0b0d017641 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:07.108087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e64d6c-3657-4131-a53b-3c0b0d017641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:07.183985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:07.186978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a7e64d6c-3657-4131-a53b-3c0b0d017641 None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:07.188339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a7e64d6c-3657-4131-a53b-3c0b0d017641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:07.309810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 706/1413] 10.222.0.83 () {58 vars in 1015 bytes} [Wed Aug 24 11:18:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:09.744303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0f67e-e919-476d-b38f-9a37f3f89dc0 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.744958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0f67e-e919-476d-b38f-9a37f3f89dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.744958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0f67e-e919-476d-b38f-9a37f3f89dc0 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.746797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:09.754185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695bf0-0b4d-488e-a8fb-34f57c91cafa None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:09.754185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695bf0-0b4d-488e-a8fb-34f57c91cafa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:09.754983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695bf0-0b4d-488e-a8fb-34f57c91cafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:09.827628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-64695bf0-0b4d-488e-a8fb-34f57c91cafa None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:09.828900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-64695bf0-0b4d-488e-a8fb-34f57c91cafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:09.888887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:09.931287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:10.239398 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0f67e-e919-476d-b38f-9a37f3f89dc0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:10.240913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-94e0f67e-e919-476d-b38f-9a37f3f89dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:10.403609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0f67e-e919-476d-b38f-9a37f3f89dc0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:10.477913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:10.522520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a60328-7552-4396-b184-a0df81ac7e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:10.523127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60328-7552-4396-b184-a0df81ac7e46 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:10.523409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60328-7552-4396-b184-a0df81ac7e46 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:10.523680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60328-7552-4396-b184-a0df81ac7e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:10.525401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-92a60328-7552-4396-b184-a0df81ac7e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:10.620982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a60328-7552-4396-b184-a0df81ac7e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:10.622720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a60328-7552-4396-b184-a0df81ac7e46 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:10.641160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 709/1416] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:11.539861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f1ae0-c964-4f4d-baa3-8d50b8878fbf None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:11.540497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f1ae0-c964-4f4d-baa3-8d50b8878fbf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:11.540497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f1ae0-c964-4f4d-baa3-8d50b8878fbf None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:11.541809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:11.548232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401eead-d01b-494f-bf8a-7af5e7f5508c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:11.548731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401eead-d01b-494f-bf8a-7af5e7f5508c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:11.548846 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401eead-d01b-494f-bf8a-7af5e7f5508c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:11.631908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9401eead-d01b-494f-bf8a-7af5e7f5508c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:11.631908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9401eead-d01b-494f-bf8a-7af5e7f5508c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:11.657463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:11.733034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:11.889950 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f1ae0-c964-4f4d-baa3-8d50b8878fbf None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:11.891437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9f0f1ae0-c964-4f4d-baa3-8d50b8878fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:12.041815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f1ae0-c964-4f4d-baa3-8d50b8878fbf None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:12.082947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:12.160233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f1be2-a126-4c10-bdfb-b486a31fd2f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:12.160782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f1be2-a126-4c10-bdfb-b486a31fd2f9 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:12.161069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f1be2-a126-4c10-bdfb-b486a31fd2f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:12.161340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f1be2-a126-4c10-bdfb-b486a31fd2f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:12.162992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-113f1be2-a126-4c10-bdfb-b486a31fd2f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:12.254388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f1be2-a126-4c10-bdfb-b486a31fd2f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:12.255991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f1be2-a126-4c10-bdfb-b486a31fd2f9 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:12.274489 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 709/1419] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:13.176190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234ef09-c652-480a-9fb5-5103fb377377 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:13.176731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234ef09-c652-480a-9fb5-5103fb377377 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:13.176731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234ef09-c652-480a-9fb5-5103fb377377 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:13.178149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:13.183854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a831608-d5f7-43e6-b619-3d93e2979e5d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:13.184296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a831608-d5f7-43e6-b619-3d93e2979e5d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:13.184411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a831608-d5f7-43e6-b619-3d93e2979e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:13.240866 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:13.264065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2a831608-d5f7-43e6-b619-3d93e2979e5d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:13.265288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2a831608-d5f7-43e6-b619-3d93e2979e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:13.364434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:13.516824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234ef09-c652-480a-9fb5-5103fb377377 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:13.518496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d234ef09-c652-480a-9fb5-5103fb377377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:13.675260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234ef09-c652-480a-9fb5-5103fb377377 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:13.688138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:13.792878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e4e17-8f90-4cbc-b109-c61132ec7f06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:13.793530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e4e17-8f90-4cbc-b109-c61132ec7f06 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:13.793812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e4e17-8f90-4cbc-b109-c61132ec7f06 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:13.794279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e4e17-8f90-4cbc-b109-c61132ec7f06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:13.796285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c30e4e17-8f90-4cbc-b109-c61132ec7f06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:13.897980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e4e17-8f90-4cbc-b109-c61132ec7f06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:13.899611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e4e17-8f90-4cbc-b109-c61132ec7f06 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:13.918482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 712/1422] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:14.796246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f770a-fa45-4517-8884-68b5fcc85d7d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.796779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f770a-fa45-4517-8884-68b5fcc85d7d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.796779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f770a-fa45-4517-8884-68b5fcc85d7d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.798252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:14.803427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cb951-0f74-42e2-9a46-dd887753ee47 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:14.803855 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cb951-0f74-42e2-9a46-dd887753ee47 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:14.803978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cb951-0f74-42e2-9a46-dd887753ee47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:14.824654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:14.883875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-796cb951-0f74-42e2-9a46-dd887753ee47 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:14.885110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-796cb951-0f74-42e2-9a46-dd887753ee47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:14.985686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:15.141741 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f770a-fa45-4517-8884-68b5fcc85d7d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:15.143193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b96f770a-fa45-4517-8884-68b5fcc85d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:15.267818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:15.308013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f770a-fa45-4517-8884-68b5fcc85d7d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:15.441149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1bf48-9565-469b-9f6e-299b19298837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:15.441760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1bf48-9565-469b-9f6e-299b19298837 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:15.442062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1bf48-9565-469b-9f6e-299b19298837 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:15.442402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1bf48-9565-469b-9f6e-299b19298837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:15.444075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-65c1bf48-9565-469b-9f6e-299b19298837 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:15.532332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:15.546725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1bf48-9565-469b-9f6e-299b19298837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:15.548668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1bf48-9565-469b-9f6e-299b19298837 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:15.568984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 712/1425] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:18:16.522887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389d850-60fd-4656-a4fc-368df5eb0338 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.523492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389d850-60fd-4656-a4fc-368df5eb0338 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.523492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389d850-60fd-4656-a4fc-368df5eb0338 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.524785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:18:16.530372 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b623028-10f0-4c94-a1c2-d525baf1da80 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:16.530788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b623028-10f0-4c94-a1c2-d525baf1da80 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:16.530909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b623028-10f0-4c94-a1c2-d525baf1da80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:16.613361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7b623028-10f0-4c94-a1c2-d525baf1da80 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:16.614522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7b623028-10f0-4c94-a1c2-d525baf1da80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:16.714963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:18:16.868770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389d850-60fd-4656-a4fc-368df5eb0338 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:16.870337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d389d850-60fd-4656-a4fc-368df5eb0338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:16.958605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:17.022931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389d850-60fd-4656-a4fc-368df5eb0338 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:17.150857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19193b55-2a2d-4fdc-917d-635a63166a99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:17.151373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19193b55-2a2d-4fdc-917d-635a63166a99 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:17.151648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19193b55-2a2d-4fdc-917d-635a63166a99 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:17.152573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19193b55-2a2d-4fdc-917d-635a63166a99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:17.154306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-19193b55-2a2d-4fdc-917d-635a63166a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:17.215885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:17.258189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19193b55-2a2d-4fdc-917d-635a63166a99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:17.259813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19193b55-2a2d-4fdc-917d-635a63166a99 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:17.278909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 715/1428] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:18.172819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d758e-4bcb-4316-9fce-48e16c7527e2 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.173433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d758e-4bcb-4316-9fce-48e16c7527e2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.173433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d758e-4bcb-4316-9fce-48e16c7527e2 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.174853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:18:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:18.180631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15da05-d5f6-44bf-a3ce-ce1fb0819cfe None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.181108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15da05-d5f6-44bf-a3ce-ce1fb0819cfe None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.181204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15da05-d5f6-44bf-a3ce-ce1fb0819cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.261059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5f15da05-d5f6-44bf-a3ce-ce1fb0819cfe None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:18.262311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5f15da05-d5f6-44bf-a3ce-ce1fb0819cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:18.360033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:18.517158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d758e-4bcb-4316-9fce-48e16c7527e2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:18.518773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-499d758e-4bcb-4316-9fce-48e16c7527e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:18.571954 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:18.672261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d758e-4bcb-4316-9fce-48e16c7527e2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:18.786261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c2139-8e98-4f55-9814-559a806025f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:18.786870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c2139-8e98-4f55-9814-559a806025f8 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:18.787146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c2139-8e98-4f55-9814-559a806025f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:18.787417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c2139-8e98-4f55-9814-559a806025f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:18.789038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cc3c2139-8e98-4f55-9814-559a806025f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:18.796955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:18.878815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c2139-8e98-4f55-9814-559a806025f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:18.880403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c2139-8e98-4f55-9814-559a806025f8 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:18.897987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 715/1431] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:19.793403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba7536-428a-4a97-b353-765de172cf84 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.793955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba7536-428a-4a97-b353-765de172cf84 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.793955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba7536-428a-4a97-b353-765de172cf84 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.795340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:19.800687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0a0ea-4bac-4954-876b-5c1290640a53 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:19.801121 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0a0ea-4bac-4954-876b-5c1290640a53 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:19.801255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0a0ea-4bac-4954-876b-5c1290640a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:19.893397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d7f0a0ea-4bac-4954-876b-5c1290640a53 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:19.895179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d7f0a0ea-4bac-4954-876b-5c1290640a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:19.983502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:20.012831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:20.164444 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba7536-428a-4a97-b353-765de172cf84 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:20.166118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5bba7536-428a-4a97-b353-765de172cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:20.185619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:20.331220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba7536-428a-4a97-b353-765de172cf84 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:20.420320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:20.446912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78389e-f6a9-4d6e-adb2-b7a773b68e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:20.447521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78389e-f6a9-4d6e-adb2-b7a773b68e99 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:20.447801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78389e-f6a9-4d6e-adb2-b7a773b68e99 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:20.448075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78389e-f6a9-4d6e-adb2-b7a773b68e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:20.449718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7d78389e-f6a9-4d6e-adb2-b7a773b68e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:20.541977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78389e-f6a9-4d6e-adb2-b7a773b68e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:20.543669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78389e-f6a9-4d6e-adb2-b7a773b68e99 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:20.562528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 718/1434] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:21.438488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf004f-a195-4706-97f2-612f15736186 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:21.439040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf004f-a195-4706-97f2-612f15736186 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:21.439040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf004f-a195-4706-97f2-612f15736186 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:21.440404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:18:21.445415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49987b70-bb3e-4e5a-ada7-71e90103fdcc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:21.445899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49987b70-bb3e-4e5a-ada7-71e90103fdcc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:21.446000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49987b70-bb3e-4e5a-ada7-71e90103fdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:21.523095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-49987b70-bb3e-4e5a-ada7-71e90103fdcc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:21.524332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-49987b70-bb3e-4e5a-ada7-71e90103fdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:21.567097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:21.640581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:21.777641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf004f-a195-4706-97f2-612f15736186 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:21.779321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ffaf004f-a195-4706-97f2-612f15736186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:21.931080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf004f-a195-4706-97f2-612f15736186 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:21.987266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:22.047842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc17bbe-76ef-4d39-a761-575c14347cad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:22.048449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc17bbe-76ef-4d39-a761-575c14347cad service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:22.048726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc17bbe-76ef-4d39-a761-575c14347cad service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:22.049001 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc17bbe-76ef-4d39-a761-575c14347cad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:22.050673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9dc17bbe-76ef-4d39-a761-575c14347cad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:22.141796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc17bbe-76ef-4d39-a761-575c14347cad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:22.143455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc17bbe-76ef-4d39-a761-575c14347cad service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:22.162198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 718/1437] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:23.026176 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230fc08-4e02-4341-ae80-39d3bbe5172d None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:23.026780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230fc08-4e02-4341-ae80-39d3bbe5172d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:23.026780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230fc08-4e02-4341-ae80-39d3bbe5172d None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:23.028102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:23.033378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad977e2-a33c-48d2-a826-15e95992d12d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:23.033833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad977e2-a33c-48d2-a826-15e95992d12d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:23.034003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad977e2-a33c-48d2-a826-15e95992d12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:23.113843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6ad977e2-a33c-48d2-a826-15e95992d12d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:23.115093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6ad977e2-a33c-48d2-a826-15e95992d12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:23.117082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:23.215922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:23.365390 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230fc08-4e02-4341-ae80-39d3bbe5172d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:23.367003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0230fc08-4e02-4341-ae80-39d3bbe5172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:23.518152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230fc08-4e02-4341-ae80-39d3bbe5172d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:23.545934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:23.641616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1adeff-b218-42f2-9a71-037343c172ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:23.642155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1adeff-b218-42f2-9a71-037343c172ff service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:23.642437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1adeff-b218-42f2-9a71-037343c172ff service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:23.642709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1adeff-b218-42f2-9a71-037343c172ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:23.644378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-da1adeff-b218-42f2-9a71-037343c172ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:23.736372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1adeff-b218-42f2-9a71-037343c172ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:23.738027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1adeff-b218-42f2-9a71-037343c172ff service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:23.756225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 721/1440] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:25.634443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e38c9-94d8-4472-aae9-411ee3cbe55d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:25.634443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e38c9-94d8-4472-aae9-411ee3cbe55d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:25.634443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e38c9-94d8-4472-aae9-411ee3cbe55d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:25.634443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:25.640907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f2ab-e8bb-4847-baee-d355352ccf97 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:25.641356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f2ab-e8bb-4847-baee-d355352ccf97 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:25.641448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f2ab-e8bb-4847-baee-d355352ccf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:25.686869 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:25.715361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e5e1f2ab-e8bb-4847-baee-d355352ccf97 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:25.716604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e5e1f2ab-e8bb-4847-baee-d355352ccf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:25.817089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:26.109956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e38c9-94d8-4472-aae9-411ee3cbe55d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:26.111545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6b0e38c9-94d8-4472-aae9-411ee3cbe55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:26.220837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e38c9-94d8-4472-aae9-411ee3cbe55d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:26.254334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:26.297694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9d112-66b5-497a-bac4-9fdd32eced58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:26.298378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d112-66b5-497a-bac4-9fdd32eced58 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:26.298663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d112-66b5-497a-bac4-9fdd32eced58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:26.298941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d112-66b5-497a-bac4-9fdd32eced58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:26.300556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a9b9d112-66b5-497a-bac4-9fdd32eced58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:26.412514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9d112-66b5-497a-bac4-9fdd32eced58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:26.414000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9d112-66b5-497a-bac4-9fdd32eced58 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:26.434192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 721/1443] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:28.620107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cbf98-c9de-431f-8d2e-1af20115ba5c None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:28.621215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cbf98-c9de-431f-8d2e-1af20115ba5c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:28.621215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cbf98-c9de-431f-8d2e-1af20115ba5c None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:28.623155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:28.628617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee018710-55a8-46e4-94fd-39cb8f55f82e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:28.639839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee018710-55a8-46e4-94fd-39cb8f55f82e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:28.639839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee018710-55a8-46e4-94fd-39cb8f55f82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:28.696898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ee018710-55a8-46e4-94fd-39cb8f55f82e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:28.698142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ee018710-55a8-46e4-94fd-39cb8f55f82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:28.711410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:28.797565 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:29.091472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cbf98-c9de-431f-8d2e-1af20115ba5c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:29.093026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-047cbf98-c9de-431f-8d2e-1af20115ba5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:29.204151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cbf98-c9de-431f-8d2e-1af20115ba5c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:29.217024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:29.280429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e1e3b-0be6-40f1-be74-db552f83e6e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:29.281040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e1e3b-0be6-40f1-be74-db552f83e6e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:29.281320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e1e3b-0be6-40f1-be74-db552f83e6e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:29.281597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e1e3b-0be6-40f1-be74-db552f83e6e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:29.283266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-985e1e3b-0be6-40f1-be74-db552f83e6e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:29.395206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e1e3b-0be6-40f1-be74-db552f83e6e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:29.396564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e1e3b-0be6-40f1-be74-db552f83e6e5 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:29.415714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 724/1446] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:31.778862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f289da-c35e-49a6-880d-e1f7a26225fa None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:31.779388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f289da-c35e-49a6-880d-e1f7a26225fa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:31.779388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f289da-c35e-49a6-880d-e1f7a26225fa None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:31.780655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:31.786180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3de0a-3002-44b9-b5a3-99d4d8e63f03 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:31.786658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3de0a-3002-44b9-b5a3-99d4d8e63f03 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:31.786844 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3de0a-3002-44b9-b5a3-99d4d8e63f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:31.837885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:31.856741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-dde3de0a-3002-44b9-b5a3-99d4d8e63f03 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:31.858031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dde3de0a-3002-44b9-b5a3-99d4d8e63f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:31.956964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:32.252597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f289da-c35e-49a6-880d-e1f7a26225fa None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:32.254202 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d4f289da-c35e-49a6-880d-e1f7a26225fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:32.360622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f289da-c35e-49a6-880d-e1f7a26225fa None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:32.401838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:32.439023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af00199-b94a-4464-ac90-e00abab61f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:32.439631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af00199-b94a-4464-ac90-e00abab61f11 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:32.439907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af00199-b94a-4464-ac90-e00abab61f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:32.440181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af00199-b94a-4464-ac90-e00abab61f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:32.441795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7af00199-b94a-4464-ac90-e00abab61f11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:32.554752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af00199-b94a-4464-ac90-e00abab61f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:32.556137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af00199-b94a-4464-ac90-e00abab61f11 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:32.576514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 724/1449] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:34.652307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d4833-9178-4ced-9f57-eda4d6cf3386 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:34.652840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d4833-9178-4ced-9f57-eda4d6cf3386 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:34.652840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d4833-9178-4ced-9f57-eda4d6cf3386 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:34.654195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:34.659616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543f0e8-d37d-4767-b9a5-ff223340b86f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:34.660063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543f0e8-d37d-4767-b9a5-ff223340b86f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:34.660171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543f0e8-d37d-4767-b9a5-ff223340b86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:34.730597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c543f0e8-d37d-4767-b9a5-ff223340b86f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:34.731860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c543f0e8-d37d-4767-b9a5-ff223340b86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:34.753367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:34.833394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:35.118114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d4833-9178-4ced-9f57-eda4d6cf3386 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:35.119793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8d4d4833-9178-4ced-9f57-eda4d6cf3386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:35.226164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d4833-9178-4ced-9f57-eda4d6cf3386 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:35.246417 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:35.305588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6c0e6-9ecb-4f20-bd52-c5b78e57ac94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:35.306321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6c0e6-9ecb-4f20-bd52-c5b78e57ac94 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:35.306608 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6c0e6-9ecb-4f20-bd52-c5b78e57ac94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:35.306887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6c0e6-9ecb-4f20-bd52-c5b78e57ac94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:35.308589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-04a6c0e6-9ecb-4f20-bd52-c5b78e57ac94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:35.427885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6c0e6-9ecb-4f20-bd52-c5b78e57ac94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:35.427885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6c0e6-9ecb-4f20-bd52-c5b78e57ac94 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:35.450266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 727/1452] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:18:37.439973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644996b6-5ce4-447a-9285-04d56b163203 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:37.440542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644996b6-5ce4-447a-9285-04d56b163203 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:37.440542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644996b6-5ce4-447a-9285-04d56b163203 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:37.441897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:37.447621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8e95-611e-4aa3-b351-254d43cc455e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:37.448096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8e95-611e-4aa3-b351-254d43cc455e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:37.448193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8e95-611e-4aa3-b351-254d43cc455e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:37.514692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:37.518114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-177d8e95-611e-4aa3-b351-254d43cc455e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:37.519400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-177d8e95-611e-4aa3-b351-254d43cc455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:37.619502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:37.908056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644996b6-5ce4-447a-9285-04d56b163203 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:37.909583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-644996b6-5ce4-447a-9285-04d56b163203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:38.016395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644996b6-5ce4-447a-9285-04d56b163203 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:38.072567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:38.100228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1777d9-9d02-4cac-9a42-4c88c6cd4605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:38.100838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1777d9-9d02-4cac-9a42-4c88c6cd4605 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:38.101120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1777d9-9d02-4cac-9a42-4c88c6cd4605 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:38.101418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1777d9-9d02-4cac-9a42-4c88c6cd4605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:38.103071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1b1777d9-9d02-4cac-9a42-4c88c6cd4605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:38.217829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1777d9-9d02-4cac-9a42-4c88c6cd4605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:38.219230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1777d9-9d02-4cac-9a42-4c88c6cd4605 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:38.237730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 727/1455] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:40.241294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d17f3-3c70-478c-b869-ab93a8c26246 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:40.241827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d17f3-3c70-478c-b869-ab93a8c26246 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:40.241827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d17f3-3c70-478c-b869-ab93a8c26246 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:40.243232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:40.249588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a83c1-ffa2-4896-a533-6595e2193707 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:40.250051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a83c1-ffa2-4896-a533-6595e2193707 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:40.250180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a83c1-ffa2-4896-a533-6595e2193707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:40.320615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-151a83c1-ffa2-4896-a533-6595e2193707 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:40.321825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-151a83c1-ffa2-4896-a533-6595e2193707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:40.351543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:40.426610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:40.711656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d17f3-3c70-478c-b869-ab93a8c26246 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:40.713218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-120d17f3-3c70-478c-b869-ab93a8c26246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:40.831510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d17f3-3c70-478c-b869-ab93a8c26246 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:40.861071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:40.908703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97180c-52bd-4ba0-91ee-f9e5c9e0b760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:40.909316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97180c-52bd-4ba0-91ee-f9e5c9e0b760 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:40.909598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97180c-52bd-4ba0-91ee-f9e5c9e0b760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:40.909899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97180c-52bd-4ba0-91ee-f9e5c9e0b760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:40.911544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6f97180c-52bd-4ba0-91ee-f9e5c9e0b760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:41.023042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97180c-52bd-4ba0-91ee-f9e5c9e0b760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:41.024503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97180c-52bd-4ba0-91ee-f9e5c9e0b760 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:41.043280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 730/1458] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:43.038414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cf47f-8e9e-48b2-a5f6-b4f285d10209 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:43.038963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cf47f-8e9e-48b2-a5f6-b4f285d10209 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:43.038963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cf47f-8e9e-48b2-a5f6-b4f285d10209 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:43.040349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:43.046748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e6b4c-a64b-4a87-96cc-31199579a34c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:43.047195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e6b4c-a64b-4a87-96cc-31199579a34c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:43.047317 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e6b4c-a64b-4a87-96cc-31199579a34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:43.116976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d86e6b4c-a64b-4a87-96cc-31199579a34c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:43.118287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d86e6b4c-a64b-4a87-96cc-31199579a34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:43.127582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:43.218291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:43.504080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cf47f-8e9e-48b2-a5f6-b4f285d10209 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:43.505644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dd8cf47f-8e9e-48b2-a5f6-b4f285d10209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:43.610285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cf47f-8e9e-48b2-a5f6-b4f285d10209 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:43.668069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:43.687260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88e74d-0c60-4f91-b429-7beda59dc665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:43.687865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88e74d-0c60-4f91-b429-7beda59dc665 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:43.688146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88e74d-0c60-4f91-b429-7beda59dc665 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:43.688410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88e74d-0c60-4f91-b429-7beda59dc665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:43.690093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1d88e74d-0c60-4f91-b429-7beda59dc665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:43.801073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88e74d-0c60-4f91-b429-7beda59dc665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:43.802540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88e74d-0c60-4f91-b429-7beda59dc665 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:43.820732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 730/1461] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:45.811373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd22f05-07b3-45e1-bf05-3e51bff3fc88 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:45.811906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd22f05-07b3-45e1-bf05-3e51bff3fc88 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:45.811906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd22f05-07b3-45e1-bf05-3e51bff3fc88 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:45.813271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:45.818718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d803-07d2-44b7-b110-bfda5c6dff3f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:45.819172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d803-07d2-44b7-b110-bfda5c6dff3f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:45.819276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d803-07d2-44b7-b110-bfda5c6dff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:45.886291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1b01d803-07d2-44b7-b110-bfda5c6dff3f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:45.887508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1b01d803-07d2-44b7-b110-bfda5c6dff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:45.928295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:45.990337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:18:46.283741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd22f05-07b3-45e1-bf05-3e51bff3fc88 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:46.285325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6bd22f05-07b3-45e1-bf05-3e51bff3fc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:46.390067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd22f05-07b3-45e1-bf05-3e51bff3fc88 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:46.429016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:46.468386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d025ee-65e0-46ac-911d-3bc6d9ec1d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:46.469104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d025ee-65e0-46ac-911d-3bc6d9ec1d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:46.469304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d025ee-65e0-46ac-911d-3bc6d9ec1d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:46.469575 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d025ee-65e0-46ac-911d-3bc6d9ec1d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:46.471214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e4d025ee-65e0-46ac-911d-3bc6d9ec1d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:46.592810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d025ee-65e0-46ac-911d-3bc6d9ec1d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:46.594289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d025ee-65e0-46ac-911d-3bc6d9ec1d50 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:46.613544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 733/1464] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:48.610371 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161003ea-6410-4f75-a96d-e9c5daa3ed1d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:48.610916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161003ea-6410-4f75-a96d-e9c5daa3ed1d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:48.610916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161003ea-6410-4f75-a96d-e9c5daa3ed1d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:48.612262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:48.618352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3b8c7-5da9-423d-ab5f-118e64b8d746 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:48.618905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3b8c7-5da9-423d-ab5f-118e64b8d746 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:48.618905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3b8c7-5da9-423d-ab5f-118e64b8d746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:48.691522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3cf3b8c7-5da9-423d-ab5f-118e64b8d746 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:48.692780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3cf3b8c7-5da9-423d-ab5f-118e64b8d746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:48.712213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:48.796532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:49.079569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161003ea-6410-4f75-a96d-e9c5daa3ed1d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:49.081149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-161003ea-6410-4f75-a96d-e9c5daa3ed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:49.211948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161003ea-6410-4f75-a96d-e9c5daa3ed1d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:49.288827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:49.296722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b2eb7-d17f-462f-8a7b-e83f7942b7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:49.297341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b2eb7-d17f-462f-8a7b-e83f7942b7dd service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:49.297626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b2eb7-d17f-462f-8a7b-e83f7942b7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:49.297927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b2eb7-d17f-462f-8a7b-e83f7942b7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:49.299657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7e0b2eb7-d17f-462f-8a7b-e83f7942b7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:49.418522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b2eb7-d17f-462f-8a7b-e83f7942b7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:49.419999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b2eb7-d17f-462f-8a7b-e83f7942b7dd service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:49.440928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 733/1467] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:18:51.415159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28297e-770c-4e74-b20d-c0cdf74c53f2 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.415685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28297e-770c-4e74-b20d-c0cdf74c53f2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.415782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28297e-770c-4e74-b20d-c0cdf74c53f2 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.417078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:18:51.422927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f37e6-3616-4fd9-9ee6-55385cf3deaf None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:51.423375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f37e6-3616-4fd9-9ee6-55385cf3deaf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:51.423517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f37e6-3616-4fd9-9ee6-55385cf3deaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:51.490726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-398f37e6-3616-4fd9-9ee6-55385cf3deaf None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:18:51.491943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-398f37e6-3616-4fd9-9ee6-55385cf3deaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:51.541768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:51.592468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:18:51.874514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28297e-770c-4e74-b20d-c0cdf74c53f2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:18:51.876126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ad28297e-770c-4e74-b20d-c0cdf74c53f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:51.982829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28297e-770c-4e74-b20d-c0cdf74c53f2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:18:52.030302 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:18:52.058443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10a411-90eb-4d56-87e7-0533c7542172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:18:52.059060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a411-90eb-4d56-87e7-0533c7542172 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:18:52.059342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a411-90eb-4d56-87e7-0533c7542172 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:18:52.059646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a411-90eb-4d56-87e7-0533c7542172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:18:52.061303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-de10a411-90eb-4d56-87e7-0533c7542172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:18:52.173609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10a411-90eb-4d56-87e7-0533c7542172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:18:52.175007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10a411-90eb-4d56-87e7-0533c7542172 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:18:52.194936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 736/1470] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:19:01.590744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=22 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:02.192596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=66 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:12.053567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758875-9875-433d-8fa7-e579b091134f None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:12.054132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758875-9875-433d-8fa7-e579b091134f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:12.054132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758875-9875-433d-8fa7-e579b091134f None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:12.055504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 735/1471] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:19:12.061261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb5b3e-3970-41ab-994e-23c8a6bee8dd None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:12.061709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb5b3e-3970-41ab-994e-23c8a6bee8dd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:12.061818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb5b3e-3970-41ab-994e-23c8a6bee8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:12.133969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bfdb5b3e-3970-41ab-994e-23c8a6bee8dd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:19:12.133969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bfdb5b3e-3970-41ab-994e-23c8a6bee8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:12.243246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 737/1472] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:19:12.551079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d02b7d-0cbb-4d30-9052-8c5c9fba8d7b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:12.551366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d02b7d-0cbb-4d30-9052-8c5c9fba8d7b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:12.551645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d02b7d-0cbb-4d30-9052-8c5c9fba8d7b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:12.553026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 736/1473] 10.222.0.83 () {54 vars in 967 bytes} [Wed Aug 24 11:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:19:12.557933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c9fb-cd50-46d3-8906-8f49b093479d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:12.558220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c9fb-cd50-46d3-8906-8f49b093479d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:12.558491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c9fb-cd50-46d3-8906-8f49b093479d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:12.638155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c421c9fb-cd50-46d3-8906-8f49b093479d None None] MFA Rules not processed for user `f1f214c2b77d4cbab2f15122ba21bf38`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:19:12.639495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c421c9fb-cd50-46d3-8906-8f49b093479d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:12.662493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:12.719043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 738/1474] 10.222.0.83 () {58 vars in 1064 bytes} [Wed Aug 24 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:19:12.726809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d02b7d-0cbb-4d30-9052-8c5c9fba8d7b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:12.728363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-06d02b7d-0cbb-4d30-9052-8c5c9fba8d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:12.837422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d02b7d-0cbb-4d30-9052-8c5c9fba8d7b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:12.912555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08033048-3c36-4cde-ae51-5e56e48f0f62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f214c2b77d4cbab2f15122ba21bf38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:12.913154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08033048-3c36-4cde-ae51-5e56e48f0f62 service glance] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:12.913436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08033048-3c36-4cde-ae51-5e56e48f0f62 service glance] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:12.913715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08033048-3c36-4cde-ae51-5e56e48f0f62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:12.915416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-08033048-3c36-4cde-ae51-5e56e48f0f62 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:12.971287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:13.028483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08033048-3c36-4cde-ae51-5e56e48f0f62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:13.029937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08033048-3c36-4cde-ae51-5e56e48f0f62 service glance] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:13.048934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 737/1475] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:19:15.109427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda56489-7808-4506-b895-2bdfbc55fbb2 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:15.110170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda56489-7808-4506-b895-2bdfbc55fbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:15.110170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda56489-7808-4506-b895-2bdfbc55fbb2 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:15.111371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 739/1476] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:19:15.116929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f860c0-a93b-4fd2-ab0c-8dae0fb2ba37 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:15.117393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f860c0-a93b-4fd2-ab0c-8dae0fb2ba37 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:15.117499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f860c0-a93b-4fd2-ab0c-8dae0fb2ba37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:15.187494 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-17f860c0-a93b-4fd2-ab0c-8dae0fb2ba37 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:19:15.188925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-17f860c0-a93b-4fd2-ab0c-8dae0fb2ba37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:15.287962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 738/1477] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:19:15.559192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda56489-7808-4506-b895-2bdfbc55fbb2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:15.560753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bda56489-7808-4506-b895-2bdfbc55fbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:15.693551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda56489-7808-4506-b895-2bdfbc55fbb2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:15.743856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:15.789351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fc5bd-ea11-4ad3-8e57-e6614770c9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:15.789994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fc5bd-ea11-4ad3-8e57-e6614770c9df admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:15.790276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fc5bd-ea11-4ad3-8e57-e6614770c9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:15.790546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fc5bd-ea11-4ad3-8e57-e6614770c9df admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:15.791192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fc5bd-ea11-4ad3-8e57-e6614770c9df admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:15.793603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-626fc5bd-ea11-4ad3-8e57-e6614770c9df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:19:15.793603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Aug 24 11:19:15.793603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: Aug 24 11:19:15.795042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fc5bd-ea11-4ad3-8e57-e6614770c9df admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:15.814134 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 740/1478] 10.222.0.83 () {56 vars in 1108 bytes} [Wed Aug 24 11:19:15 2022] GET /identity/v3/projects => generated 2553 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:19:17.839311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af1d6d-428c-48b5-8d4b-810f7f0e4fae None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:17.839854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af1d6d-428c-48b5-8d4b-810f7f0e4fae None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:17.839854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af1d6d-428c-48b5-8d4b-810f7f0e4fae None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:17.841256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 739/1479] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:19:17.848434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccb2a6-691a-45fc-8932-39ad55399627 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:17.848910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccb2a6-691a-45fc-8932-39ad55399627 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:17.849031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccb2a6-691a-45fc-8932-39ad55399627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:17.932511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-92ccb2a6-691a-45fc-8932-39ad55399627 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:19:17.933753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-92ccb2a6-691a-45fc-8932-39ad55399627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:18.033059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 741/1480] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:19:18.327957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af1d6d-428c-48b5-8d4b-810f7f0e4fae None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:18.329519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-17af1d6d-428c-48b5-8d4b-810f7f0e4fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:18.339937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:18.482166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af1d6d-428c-48b5-8d4b-810f7f0e4fae None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:18.575438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:18.730257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ecbef-cfb3-409e-96c5-1685a772293b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:18.730860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ecbef-cfb3-409e-96c5-1685a772293b service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:18.731140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ecbef-cfb3-409e-96c5-1685a772293b service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:18.731412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ecbef-cfb3-409e-96c5-1685a772293b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:18.733027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-af7ecbef-cfb3-409e-96c5-1685a772293b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:18.824170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ecbef-cfb3-409e-96c5-1685a772293b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:18.825829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ecbef-cfb3-409e-96c5-1685a772293b service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:18.844615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 740/1481] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:19:20.972009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057f04-8ff6-4705-ba8d-bdf8e6c9ca69 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:20.972607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057f04-8ff6-4705-ba8d-bdf8e6c9ca69 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:20.972607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057f04-8ff6-4705-ba8d-bdf8e6c9ca69 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:20.973915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 742/1482] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:19:20.980185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f258e1-564f-40bf-b63f-8e5675edd45c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:20.980694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f258e1-564f-40bf-b63f-8e5675edd45c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:20.980818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f258e1-564f-40bf-b63f-8e5675edd45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:21.047158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f8f258e1-564f-40bf-b63f-8e5675edd45c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:19:21.048369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f8f258e1-564f-40bf-b63f-8e5675edd45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:21.095111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:21.145239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 741/1483] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:19:21.431689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057f04-8ff6-4705-ba8d-bdf8e6c9ca69 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:21.433295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-60057f04-8ff6-4705-ba8d-bdf8e6c9ca69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:21.457195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:21.538240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057f04-8ff6-4705-ba8d-bdf8e6c9ca69 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:21.615015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99054e-2f5c-480a-bf92-e0309814b376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:21.615579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99054e-2f5c-480a-bf92-e0309814b376 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:21.615821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99054e-2f5c-480a-bf92-e0309814b376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:21.616085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99054e-2f5c-480a-bf92-e0309814b376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:21.617710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cf99054e-2f5c-480a-bf92-e0309814b376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:21.694440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:21.730342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99054e-2f5c-480a-bf92-e0309814b376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:21.731791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99054e-2f5c-480a-bf92-e0309814b376 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:21.749649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 743/1484] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:19:23.748848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570284e-72e7-435b-9021-40deafdf0c02 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:23.749378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570284e-72e7-435b-9021-40deafdf0c02 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:23.749378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570284e-72e7-435b-9021-40deafdf0c02 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:23.750730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 742/1485] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:19:23.756162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c389dc-e1e2-498f-b39f-b76cc3b7ae4e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:23.756602 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c389dc-e1e2-498f-b39f-b76cc3b7ae4e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:23.756712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c389dc-e1e2-498f-b39f-b76cc3b7ae4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:23.823990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a0c389dc-e1e2-498f-b39f-b76cc3b7ae4e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:19:23.825212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a0c389dc-e1e2-498f-b39f-b76cc3b7ae4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:23.921658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 744/1486] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:19:24.211438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570284e-72e7-435b-9021-40deafdf0c02 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:24.213610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a570284e-72e7-435b-9021-40deafdf0c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:24.333740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570284e-72e7-435b-9021-40deafdf0c02 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:24.413711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:24.415664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62042115-356a-476e-8c87-274d9dfae290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:24.416371 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62042115-356a-476e-8c87-274d9dfae290 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:24.416651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62042115-356a-476e-8c87-274d9dfae290 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:24.416921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62042115-356a-476e-8c87-274d9dfae290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:24.418640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-62042115-356a-476e-8c87-274d9dfae290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:24.532774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62042115-356a-476e-8c87-274d9dfae290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:24.534227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62042115-356a-476e-8c87-274d9dfae290 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:24.554969 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 743/1487] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:19:26.567133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38046c68-87a1-4e8f-9fb5-8979c44e65c7 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:26.567689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38046c68-87a1-4e8f-9fb5-8979c44e65c7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:26.567689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38046c68-87a1-4e8f-9fb5-8979c44e65c7 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:26.569078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 745/1488] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:19:26.575142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0acb7d-a812-4a58-b8e6-bd74fde18611 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:26.575630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0acb7d-a812-4a58-b8e6-bd74fde18611 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:26.575769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0acb7d-a812-4a58-b8e6-bd74fde18611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:26.651516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2b0acb7d-a812-4a58-b8e6-bd74fde18611 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:19:26.652763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2b0acb7d-a812-4a58-b8e6-bd74fde18611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:26.705753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:26.750875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 744/1489] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:19:27.042005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38046c68-87a1-4e8f-9fb5-8979c44e65c7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:27.043541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-38046c68-87a1-4e8f-9fb5-8979c44e65c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:27.081032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:27.148877 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38046c68-87a1-4e8f-9fb5-8979c44e65c7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:27.225721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f37262-efab-47da-b5d2-b2f544fdb4a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:27.226357 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f37262-efab-47da-b5d2-b2f544fdb4a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:27.226639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f37262-efab-47da-b5d2-b2f544fdb4a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:27.226911 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f37262-efab-47da-b5d2-b2f544fdb4a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:27.228525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c7f37262-efab-47da-b5d2-b2f544fdb4a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:27.315171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:27.345104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f37262-efab-47da-b5d2-b2f544fdb4a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:27.346601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f37262-efab-47da-b5d2-b2f544fdb4a4 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:27.365509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 746/1490] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:19:29.716874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3cbf-75c2-4d07-b432-ab2426d8d419 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:29.717437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3cbf-75c2-4d07-b432-ab2426d8d419 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:29.717437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3cbf-75c2-4d07-b432-ab2426d8d419 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:29.718722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 745/1491] 10.222.0.83 () {54 vars in 927 bytes} [Wed Aug 24 11:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:19:29.724324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d63be0-601f-4914-b187-03770b3921fb None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:29.724806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d63be0-601f-4914-b187-03770b3921fb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:29.724916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d63be0-601f-4914-b187-03770b3921fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:29.792901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-12d63be0-601f-4914-b187-03770b3921fb None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:19:29.794176 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-12d63be0-601f-4914-b187-03770b3921fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:29.893236 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 747/1492] 10.222.0.83 () {58 vars in 1024 bytes} [Wed Aug 24 11:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:19:30.184442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3cbf-75c2-4d07-b432-ab2426d8d419 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:19:30.186049 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f38b3cbf-75c2-4d07-b432-ab2426d8d419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:30.290657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3cbf-75c2-4d07-b432-ab2426d8d419 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:19:30.363934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fcb64e-4741-436e-a875-22005430a8b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:19:30.364531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fcb64e-4741-436e-a875-22005430a8b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:19:30.364806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fcb64e-4741-436e-a875-22005430a8b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:19:30.365076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fcb64e-4741-436e-a875-22005430a8b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:19:30.366695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-44fcb64e-4741-436e-a875-22005430a8b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:19:30.376336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:30.481174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fcb64e-4741-436e-a875-22005430a8b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:19:30.482563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fcb64e-4741-436e-a875-22005430a8b9 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:19:30.500133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 746/1493] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:19:39.891477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=82 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:19:40.497834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=50 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:20:59.633139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968630b-c8d0-4757-b061-b391a75970f1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:20:59.634031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968630b-c8d0-4757-b061-b391a75970f1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:20:59.634031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968630b-c8d0-4757-b061-b391a75970f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:20:59.721882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6968630b-c8d0-4757-b061-b391a75970f1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:20:59.723133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6968630b-c8d0-4757-b061-b391a75970f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:20:59.827854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 748/1494] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:20:59.836419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd021d9-8911-458c-a033-68346483e30c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:20:59.837012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd021d9-8911-458c-a033-68346483e30c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:20:59.837116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd021d9-8911-458c-a033-68346483e30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:20:59.924140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-dfd021d9-8911-458c-a033-68346483e30c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:20:59.925314 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dfd021d9-8911-458c-a033-68346483e30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:00.029924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 747/1495] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:21:00.045487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968630b-c8d0-4757-b061-b391a75970f1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:00.046854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6968630b-c8d0-4757-b061-b391a75970f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:00.059049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968630b-c8d0-4757-b061-b391a75970f1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:00.130980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:00.166618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:00.167267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:00.167541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:00.167870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:00.168741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:21:00.169234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:00.170902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:21:00.170902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Aug 24 11:21:00.170902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: Aug 24 11:21:00.171301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:00.184878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 749/1496] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:21:00.193649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd021d9-8911-458c-a033-68346483e30c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:00.203702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd021d9-8911-458c-a033-68346483e30c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:00.265608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:00.301363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:00.302087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:00.302354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:00.302537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:00.303164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:00.304937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:21:00.304937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Aug 24 11:21:00.304937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: Aug 24 11:21:00.306175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:00.457955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 748/1497] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:21:00 2022] POST /identity/v3/projects => generated 379 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:21:00.462170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:00.463724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:00.499710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c44c5-b5ed-4c49-b365-e8942b048f47 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:00.600063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:00.600683 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:00.600957 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:00.601230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:00.601961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:00.603047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:21:00.603047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Aug 24 11:21:00.603047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: Aug 24 11:21:00.605787 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:00.770809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 750/1498] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:21:00 2022] POST /identity/v3/users => generated 290 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:21:00.774918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:00.776421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:00.840012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4b5fb-574f-4ef0-a5a7-97e91f1e309b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:00.927651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:00.938394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:00.939354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:00.939354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:00.939561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:00.940126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:00.941119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:21:00.941119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Aug 24 11:21:00.941119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: Aug 24 11:21:00.942055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:00.954708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 749/1499] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:21:00.959172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:00.960740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:00.973672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa6f72-3727-4069-b719-1fa2f71e223d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:01.001101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:01.199051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476a7aa-5ea1-490f-b245-0a5d58c008f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:01.199658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476a7aa-5ea1-490f-b245-0a5d58c008f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:01.199933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476a7aa-5ea1-490f-b245-0a5d58c008f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:01.200198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476a7aa-5ea1-490f-b245-0a5d58c008f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:01.200861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476a7aa-5ea1-490f-b245-0a5d58c008f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:01.201916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-5476a7aa-5ea1-490f-b245-0a5d58c008f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:21:01.201916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Aug 24 11:21:01.201916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: Aug 24 11:21:01.202794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476a7aa-5ea1-490f-b245-0a5d58c008f9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:01.217329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 751/1500] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:21:01.222180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:01.223719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:01.258813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c160b92-48b0-46d5-891e-bd994b05b7d0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:01.361060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb12e0-975d-4c76-a5c7-61fb90a338bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:01.361724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb12e0-975d-4c76-a5c7-61fb90a338bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:01.362048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb12e0-975d-4c76-a5c7-61fb90a338bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:01.362310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb12e0-975d-4c76-a5c7-61fb90a338bf admin admin] PATH_INFO: `/v3/projects/c386b641cbca4b37bfda858dc21c94f4/users/a1ee5c45af2443bf877caed81fc6c1b0/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:01.384848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb12e0-975d-4c76-a5c7-61fb90a338bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c386b641cbca4b37bfda858dc21c94f4, user_id=a1ee5c45af2443bf877caed81fc6c1b0, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:01.386417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-95eb12e0-975d-4c76-a5c7-61fb90a338bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:21:01.386417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Aug 24 11:21:01.386417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: Aug 24 11:21:01.388377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb12e0-975d-4c76-a5c7-61fb90a338bf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:01.484215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 750/1501] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:21:01 2022] PUT /identity/v3/projects/c386b641cbca4b37bfda858dc21c94f4/users/a1ee5c45af2443bf877caed81fc6c1b0/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:21:01.496457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da706c-ceb7-4df2-9b2f-737c615406ce None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:01.497778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da706c-ceb7-4df2-9b2f-737c615406ce None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:01.497778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da706c-ceb7-4df2-9b2f-737c615406ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:01.558199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-57da706c-ceb7-4df2-9b2f-737c615406ce None None] MFA Rules not processed for user `a1ee5c45af2443bf877caed81fc6c1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:21:01.559467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-57da706c-ceb7-4df2-9b2f-737c615406ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:01.647184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 752/1502] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:21:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:21:01.660862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b38e5a-44f3-4e9b-b802-6e8ab4bad3d7 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:01.661145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b38e5a-44f3-4e9b-b802-6e8ab4bad3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:01.661420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b38e5a-44f3-4e9b-b802-6e8ab4bad3d7 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:01.662809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 751/1503] 10.222.0.83 () {54 vars in 964 bytes} [Wed Aug 24 11:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:21:01.667723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbd81f-d02d-4759-86b9-1f2bbffc3b92 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:01.668008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbd81f-d02d-4759-86b9-1f2bbffc3b92 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:01.668763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbd81f-d02d-4759-86b9-1f2bbffc3b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:01.707795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:01.752536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-dafbd81f-d02d-4759-86b9-1f2bbffc3b92 None None] MFA Rules not processed for user `f9dd6f187dd84f04866da64732b242d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:21:01.753675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dafbd81f-d02d-4759-86b9-1f2bbffc3b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:01.830165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 753/1504] 10.222.0.83 () {58 vars in 1061 bytes} [Wed Aug 24 11:21:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:21:01.837891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b38e5a-44f3-4e9b-b802-6e8ab4bad3d7 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:01.839411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-01b38e5a-44f3-4e9b-b802-6e8ab4bad3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:01.890682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:01.946432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b38e5a-44f3-4e9b-b802-6e8ab4bad3d7 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:02.021987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:02.022525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:02.022802 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:02.023070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:02.024652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:02.129700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:02.131171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:02.134981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:02.150016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 752/1505] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:21:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:21:02.198674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62b990-992f-49ce-bfd6-791ce193e399 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:02.198969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62b990-992f-49ce-bfd6-791ce193e399 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:02.202666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62b990-992f-49ce-bfd6-791ce193e399 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:02.202666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 754/1506] 10.222.0.83 () {52 vars in 830 bytes} [Wed Aug 24 11:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:21:02.204798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:02.206128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:02.314687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b329c-2649-4b68-b2cd-47059d56b5da service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:02.413143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:02.413698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:02.414005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:02.414278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] PATH_INFO: `/v3/users/a1ee5c45af2443bf877caed81fc6c1b0` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:02.428438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ee5c45af2443bf877caed81fc6c1b0)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:02.429588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:21:02.429588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Aug 24 11:21:02.429588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: Aug 24 11:21:02.430743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:02.442143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:02.664041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ee5c45af2443bf877caed81fc6c1b0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:02.664491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ee5c45af2443bf877caed81fc6c1b0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:02.664906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ee5c45af2443bf877caed81fc6c1b0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:02.733433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ee5c45af2443bf877caed81fc6c1b0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:02.748553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ee5c45af2443bf877caed81fc6c1b0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:02.778483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 753/1507] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:21:02 2022] DELETE /identity/v3/users/a1ee5c45af2443bf877caed81fc6c1b0 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:21:02.782861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbfee2-9d78-49d0-bd5c-29d134f4f2dc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:02.783142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbfee2-9d78-49d0-bd5c-29d134f4f2dc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:02.783412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbfee2-9d78-49d0-bd5c-29d134f4f2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:02.842639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2fcbfee2-9d78-49d0-bd5c-29d134f4f2dc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:21:02.843859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2fcbfee2-9d78-49d0-bd5c-29d134f4f2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:02.920433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:02.932783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 755/1508] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:21:02.942864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:02.944171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:03.046970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10b366-fede-4df2-a7d3-961296907ff4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:03.106047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:03.126398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041ec90-6faa-4ae0-8068-b6b1b165e973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:03.127038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041ec90-6faa-4ae0-8068-b6b1b165e973 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:03.127315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041ec90-6faa-4ae0-8068-b6b1b165e973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:03.127588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041ec90-6faa-4ae0-8068-b6b1b165e973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:03.129221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d041ec90-6faa-4ae0-8068-b6b1b165e973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:03.241648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041ec90-6faa-4ae0-8068-b6b1b165e973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:03.243101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041ec90-6faa-4ae0-8068-b6b1b165e973 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:03.262215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 754/1509] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:21:03.882489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbfee2-9d78-49d0-bd5c-29d134f4f2dc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:21:03.883819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2fcbfee2-9d78-49d0-bd5c-29d134f4f2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:21:03.895862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbfee2-9d78-49d0-bd5c-29d134f4f2dc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:21:04.005438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:21:04.006102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:21:04.006425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:21:04.006697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] PATH_INFO: `/v3/projects/c386b641cbca4b37bfda858dc21c94f4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:21:04.016851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c386b641cbca4b37bfda858dc21c94f4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:21:04.017900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:21:04.017900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Aug 24 11:21:04.017900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: Aug 24 11:21:04.019094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:21:04.135081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c386b641cbca4b37bfda858dc21c94f4', 'request_id': 'req-57853246-6cd5-4e3c-82d9-f24dc7c400fb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:04.141270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c386b641cbca4b37bfda858dc21c94f4', 'request_id': 'req-57853246-6cd5-4e3c-82d9-f24dc7c400fb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:04.200230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c386b641cbca4b37bfda858dc21c94f4', 'request_id': 'req-57853246-6cd5-4e3c-82d9-f24dc7c400fb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:04.232042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] The token cache is being invalidate because project c386b641cbca4b37bfda858dc21c94f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:21:04.232324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:04.233411 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57853246-6cd5-4e3c-82d9-f24dc7c400fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:21:04.234792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 756/1510] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:21:03 2022] DELETE /identity/v3/projects/c386b641cbca4b37bfda858dc21c94f4 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:21:13.259878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=62 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:21:14.229979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:22:40.161821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c0ea4-9164-44e4-8ad3-7850a6008d15 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:40.162444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c0ea4-9164-44e4-8ad3-7850a6008d15 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:40.162444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c0ea4-9164-44e4-8ad3-7850a6008d15 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:40.163771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:22:40.169155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccffad-eb3b-4767-ada7-694c560a844c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:40.169586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccffad-eb3b-4767-ada7-694c560a844c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:40.169731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccffad-eb3b-4767-ada7-694c560a844c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:40.251843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0dccffad-eb3b-4767-ada7-694c560a844c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:22:40.253060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0dccffad-eb3b-4767-ada7-694c560a844c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:40.358873 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:22:40.514112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c0ea4-9164-44e4-8ad3-7850a6008d15 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:40.516276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-424c0ea4-9164-44e4-8ad3-7850a6008d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:40.645905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c0ea4-9164-44e4-8ad3-7850a6008d15 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:40.651891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f953f65-6301-4faf-95e6-ab75eb763b56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:40.652476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f953f65-6301-4faf-95e6-ab75eb763b56 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:40.652754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f953f65-6301-4faf-95e6-ab75eb763b56 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:40.653022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f953f65-6301-4faf-95e6-ab75eb763b56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:40.662375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f953f65-6301-4faf-95e6-ab75eb763b56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:40.663969 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f953f65-6301-4faf-95e6-ab75eb763b56 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:40.674502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 756/1513] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:22:41.526767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefa950-b245-4e77-9a17-06d3730ca5a1 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:41.527331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefa950-b245-4e77-9a17-06d3730ca5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:41.527331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefa950-b245-4e77-9a17-06d3730ca5a1 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:41.528670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:22:41.534430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beef53-5afb-49f4-ba83-da2f6915789c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:41.534888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beef53-5afb-49f4-ba83-da2f6915789c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:41.534987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beef53-5afb-49f4-ba83-da2f6915789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:41.554362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-82beef53-5afb-49f4-ba83-da2f6915789c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:22:41.555558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-82beef53-5afb-49f4-ba83-da2f6915789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:41.563486 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:22:41.716218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefa950-b245-4e77-9a17-06d3730ca5a1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:41.731210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefa950-b245-4e77-9a17-06d3730ca5a1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:41.736894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efb0f0-9926-48a4-a2fc-d67248f83c1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:41.737514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efb0f0-9926-48a4-a2fc-d67248f83c1b service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:41.737786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efb0f0-9926-48a4-a2fc-d67248f83c1b service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:41.738099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efb0f0-9926-48a4-a2fc-d67248f83c1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:41.747149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efb0f0-9926-48a4-a2fc-d67248f83c1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:41.748748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efb0f0-9926-48a4-a2fc-d67248f83c1b service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:41.758983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 759/1516] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:22:42.088240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefdc66-2e18-4e4b-9de3-eb59a4bc66f1 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:42.088555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefdc66-2e18-4e4b-9de3-eb59a4bc66f1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:42.088847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefdc66-2e18-4e4b-9de3-eb59a4bc66f1 None None] PATH_INFO: `/v3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:42.090242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 758/1517] 10.222.0.102 () {54 vars in 926 bytes} [Wed Aug 24 11:22:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:22:42.097837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223f0a4-f79f-4832-be1c-f07db0013899 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:42.098145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223f0a4-f79f-4832-be1c-f07db0013899 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:42.098416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223f0a4-f79f-4832-be1c-f07db0013899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:42.158268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c223f0a4-f79f-4832-be1c-f07db0013899 None None] MFA Rules not processed for user `eb76ad667a744126ab00b38c346e9017`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:22:42.159480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c223f0a4-f79f-4832-be1c-f07db0013899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:42.177727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:22:42.231169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 760/1518] 10.222.0.102 () {58 vars in 1016 bytes} [Wed Aug 24 11:22:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:22:51.549627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=56 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:22:52.228237 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=23 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:22:52.327222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951039-9678-49b6-a274-fa3cff3ef4fa None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:52.327517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951039-9678-49b6-a274-fa3cff3ef4fa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:52.327796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951039-9678-49b6-a274-fa3cff3ef4fa None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:52.329171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 759/1519] 10.222.0.83 () {54 vars in 961 bytes} [Wed Aug 24 11:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:22:52.335090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc3e9c-8bbc-4939-887e-ce2315cf0ac7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:52.335381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc3e9c-8bbc-4939-887e-ce2315cf0ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:52.335661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc3e9c-8bbc-4939-887e-ce2315cf0ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:52.355478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b8fc3e9c-8bbc-4939-887e-ce2315cf0ac7 None None] MFA Rules not processed for user `eb76ad667a744126ab00b38c346e9017`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:22:52.356689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b8fc3e9c-8bbc-4939-887e-ce2315cf0ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:52.365561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 761/1520] 10.222.0.83 () {58 vars in 1058 bytes} [Wed Aug 24 11:22:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:22:52.373164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32951039-9678-49b6-a274-fa3cff3ef4fa None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:52.387691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32951039-9678-49b6-a274-fa3cff3ef4fa None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:52.393308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6623d9e-f5b4-4167-a372-419f943673fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:52.393925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6623d9e-f5b4-4167-a372-419f943673fd service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:52.394222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6623d9e-f5b4-4167-a372-419f943673fd service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:52.394485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6623d9e-f5b4-4167-a372-419f943673fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:52.403003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6623d9e-f5b4-4167-a372-419f943673fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:52.404585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6623d9e-f5b4-4167-a372-419f943673fd service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:52.415028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 760/1521] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:22:53.016792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533a3a-2227-4239-9326-3e73b8b16d2c None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:53.017065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533a3a-2227-4239-9326-3e73b8b16d2c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:53.017962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533a3a-2227-4239-9326-3e73b8b16d2c None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:53.019430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:22:53.024473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444415c3-ed41-49d0-95ab-5fe4c23b8d99 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:53.024927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444415c3-ed41-49d0-95ab-5fe4c23b8d99 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:53.025040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444415c3-ed41-49d0-95ab-5fe4c23b8d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:53.045062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-444415c3-ed41-49d0-95ab-5fe4c23b8d99 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:22:53.046308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-444415c3-ed41-49d0-95ab-5fe4c23b8d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:53.054352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:22:53.210817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2533a3a-2227-4239-9326-3e73b8b16d2c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:53.224912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2533a3a-2227-4239-9326-3e73b8b16d2c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:53.231204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d8975-c8de-428c-af02-b9c12bc15b19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:53.231816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d8975-c8de-428c-af02-b9c12bc15b19 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:53.232071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d8975-c8de-428c-af02-b9c12bc15b19 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:53.232344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d8975-c8de-428c-af02-b9c12bc15b19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:53.242108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d8975-c8de-428c-af02-b9c12bc15b19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:53.243738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d8975-c8de-428c-af02-b9c12bc15b19 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:53.253961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 763/1524] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:22:54.080365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715fa46-ed93-4ffc-bf14-0b63d3a2ff49 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:54.080945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715fa46-ed93-4ffc-bf14-0b63d3a2ff49 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:54.080945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715fa46-ed93-4ffc-bf14-0b63d3a2ff49 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:54.082332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:22:54.087044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bb74d-d20e-4357-99aa-aa564ad539bd None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:54.087329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bb74d-d20e-4357-99aa-aa564ad539bd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:54.087637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bb74d-d20e-4357-99aa-aa564ad539bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:54.106510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-915bb74d-d20e-4357-99aa-aa564ad539bd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:22:54.107690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-915bb74d-d20e-4357-99aa-aa564ad539bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:54.116229 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:22:54.266808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715fa46-ed93-4ffc-bf14-0b63d3a2ff49 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:54.281483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715fa46-ed93-4ffc-bf14-0b63d3a2ff49 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:54.287022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a055be-ca19-44f4-aa77-78cc4ed4f290 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:54.287620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a055be-ca19-44f4-aa77-78cc4ed4f290 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:54.287899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a055be-ca19-44f4-aa77-78cc4ed4f290 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:54.288172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a055be-ca19-44f4-aa77-78cc4ed4f290 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:54.296908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a055be-ca19-44f4-aa77-78cc4ed4f290 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:54.299047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a055be-ca19-44f4-aa77-78cc4ed4f290 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:54.308474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 763/1527] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:22:58.279275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5deae2-8307-4d61-a090-398c565b200a None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:58.279841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5deae2-8307-4d61-a090-398c565b200a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:58.279941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5deae2-8307-4d61-a090-398c565b200a None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:58.281945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:22:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:22:58.287897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e4608-fdd6-4522-be11-498ac0029f66 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:58.288364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e4608-fdd6-4522-be11-498ac0029f66 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:58.288459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e4608-fdd6-4522-be11-498ac0029f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:58.308394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-950e4608-fdd6-4522-be11-498ac0029f66 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:22:58.309598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-950e4608-fdd6-4522-be11-498ac0029f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:58.317472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:22:58.607717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5deae2-8307-4d61-a090-398c565b200a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:22:58.609456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9f5deae2-8307-4d61-a090-398c565b200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:22:58.691624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5deae2-8307-4d61-a090-398c565b200a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:22:58.695291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b22da7-c9a5-4dea-8d06-a2497af89f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:22:58.695902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b22da7-c9a5-4dea-8d06-a2497af89f40 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:22:58.696175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b22da7-c9a5-4dea-8d06-a2497af89f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:22:58.696447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b22da7-c9a5-4dea-8d06-a2497af89f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:22:58.705568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b22da7-c9a5-4dea-8d06-a2497af89f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:22:58.707098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b22da7-c9a5-4dea-8d06-a2497af89f40 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:22:58.716735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 766/1530] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:00.625843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac9dd0-a1c6-4d19-9c57-7a508aec035b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:00.626379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac9dd0-a1c6-4d19-9c57-7a508aec035b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:00.626379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac9dd0-a1c6-4d19-9c57-7a508aec035b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:00.627679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:00.636796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92cfa-8113-425f-ba27-7d80f3f6b4e7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:00.637303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92cfa-8113-425f-ba27-7d80f3f6b4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:00.637399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92cfa-8113-425f-ba27-7d80f3f6b4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:00.661525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8ac92cfa-8113-425f-ba27-7d80f3f6b4e7 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:00.662784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8ac92cfa-8113-425f-ba27-7d80f3f6b4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:00.670957 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:00.971671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac9dd0-a1c6-4d19-9c57-7a508aec035b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:00.973381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-51ac9dd0-a1c6-4d19-9c57-7a508aec035b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:00.986346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac9dd0-a1c6-4d19-9c57-7a508aec035b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:00.989849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdab9e-5344-4f52-aa00-2181291c0f37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:00.990470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdab9e-5344-4f52-aa00-2181291c0f37 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:00.990747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdab9e-5344-4f52-aa00-2181291c0f37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:00.991021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdab9e-5344-4f52-aa00-2181291c0f37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:00.999782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdab9e-5344-4f52-aa00-2181291c0f37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:01.001205 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdab9e-5344-4f52-aa00-2181291c0f37 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:01.010665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 766/1533] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:23:04.997430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804df54f-d986-4185-909e-17da9da32ad1 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:04.998025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804df54f-d986-4185-909e-17da9da32ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:04.998025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804df54f-d986-4185-909e-17da9da32ad1 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:04.999276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:05.004530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9399f76-53e9-4ec5-9954-c0ec5518f0a6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:05.004974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9399f76-53e9-4ec5-9954-c0ec5518f0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:05.005081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9399f76-53e9-4ec5-9954-c0ec5518f0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:05.024800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e9399f76-53e9-4ec5-9954-c0ec5518f0a6 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:05.026056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e9399f76-53e9-4ec5-9954-c0ec5518f0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:05.034817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:05.588526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804df54f-d986-4185-909e-17da9da32ad1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:05.590330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-804df54f-d986-4185-909e-17da9da32ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:05.668923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804df54f-d986-4185-909e-17da9da32ad1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:05.672470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608e7ba-99d4-49a4-8c0e-c5dda0a630f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:05.673074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608e7ba-99d4-49a4-8c0e-c5dda0a630f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:05.673343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608e7ba-99d4-49a4-8c0e-c5dda0a630f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:05.673652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608e7ba-99d4-49a4-8c0e-c5dda0a630f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:05.682656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608e7ba-99d4-49a4-8c0e-c5dda0a630f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:05.684090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608e7ba-99d4-49a4-8c0e-c5dda0a630f1 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:05.694009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 769/1536] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:07.618885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fc78c-bb91-4c0e-bcd4-14827341dd31 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:07.619423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fc78c-bb91-4c0e-bcd4-14827341dd31 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:07.619423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fc78c-bb91-4c0e-bcd4-14827341dd31 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:07.620774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:07.626109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410f2a4-ff08-4c35-b182-f464580e787c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:07.627118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410f2a4-ff08-4c35-b182-f464580e787c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:07.627297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410f2a4-ff08-4c35-b182-f464580e787c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:07.646549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b410f2a4-ff08-4c35-b182-f464580e787c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:07.647758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b410f2a4-ff08-4c35-b182-f464580e787c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:07.655903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:08.203682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fc78c-bb91-4c0e-bcd4-14827341dd31 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:08.205387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-235fc78c-bb91-4c0e-bcd4-14827341dd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:08.218081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fc78c-bb91-4c0e-bcd4-14827341dd31 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:08.222069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f562c28-11ee-4679-93b9-ab5353d98787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:08.222652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f562c28-11ee-4679-93b9-ab5353d98787 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:08.222926 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f562c28-11ee-4679-93b9-ab5353d98787 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:08.223192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f562c28-11ee-4679-93b9-ab5353d98787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:08.231779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f562c28-11ee-4679-93b9-ab5353d98787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:08.233189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f562c28-11ee-4679-93b9-ab5353d98787 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:08.242947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 769/1539] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:17.641721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=88 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:18.238878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=33 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:36.712364 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bb229-713e-410b-98c8-1075369999e0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:36.713668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bb229-713e-410b-98c8-1075369999e0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:36.714228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bb229-713e-410b-98c8-1075369999e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:36.738583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-075bb229-713e-410b-98c8-1075369999e0 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:36.739822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-075bb229-713e-410b-98c8-1075369999e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:36.753541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 771/1540] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:36.761985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b824c1-6ad6-4b9a-8800-6449c62fa32e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:36.763193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b824c1-6ad6-4b9a-8800-6449c62fa32e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:36.763713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b824c1-6ad6-4b9a-8800-6449c62fa32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:36.763835 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0006b9-7d44-4447-acdc-314829b28693 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:36.764070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0006b9-7d44-4447-acdc-314829b28693 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:36.764372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0006b9-7d44-4447-acdc-314829b28693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:36.790082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fd0006b9-7d44-4447-acdc-314829b28693 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:36.791612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fd0006b9-7d44-4447-acdc-314829b28693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:36.798546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-69b824c1-6ad6-4b9a-8800-6449c62fa32e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:36.800229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-69b824c1-6ad6-4b9a-8800-6449c62fa32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:36.815178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 770/1541] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:36.820989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 772/1542] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:36.825629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b824c1-6ad6-4b9a-8800-6449c62fa32e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:36.828401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776654b3-a185-4fde-8005-18baa2525954 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:36.828697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776654b3-a185-4fde-8005-18baa2525954 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:36.828978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776654b3-a185-4fde-8005-18baa2525954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:36.839412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b824c1-6ad6-4b9a-8800-6449c62fa32e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:36.857469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:36.859126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-776654b3-a185-4fde-8005-18baa2525954 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:36.859622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:36.860140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:36.860379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-776654b3-a185-4fde-8005-18baa2525954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:36.860843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:36.861816 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:36.862468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:36.863709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:24:36.863709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Aug 24 11:24:36.863709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: Aug 24 11:24:36.865077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:36.872359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 773/1543] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:24:36.876856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776654b3-a185-4fde-8005-18baa2525954 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:36.888893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776654b3-a185-4fde-8005-18baa2525954 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:36.892102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 771/1544] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:36.896772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f70edb-b3f3-4a5f-98ce-4ef178865fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:36.901378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f70edb-b3f3-4a5f-98ce-4ef178865fef admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:36.901378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f70edb-b3f3-4a5f-98ce-4ef178865fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:36.901378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f70edb-b3f3-4a5f-98ce-4ef178865fef admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:36.901378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f70edb-b3f3-4a5f-98ce-4ef178865fef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:36.901378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f70edb-b3f3-4a5f-98ce-4ef178865fef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:36.901790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:36.901903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f70edb-b3f3-4a5f-98ce-4ef178865fef admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:36.917937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5b0bc-20f3-4bbc-ad0f-d57e34b9380c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:36.919690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 774/1545] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:36.927766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f70edb-b3f3-4a5f-98ce-4ef178865fef admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:36.928726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccf2c1-6cde-48e1-beac-ff1840f0db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:36.929792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccf2c1-6cde-48e1-beac-ff1840f0db00 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:36.930552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccf2c1-6cde-48e1-beac-ff1840f0db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:36.931721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccf2c1-6cde-48e1-beac-ff1840f0db00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:36.932616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccf2c1-6cde-48e1-beac-ff1840f0db00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:36.935047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccf2c1-6cde-48e1-beac-ff1840f0db00 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:36.942898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f70edb-b3f3-4a5f-98ce-4ef178865fef admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:36.948349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440af8a-f8f8-4a08-9f02-8a72188c6df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:36.948977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440af8a-f8f8-4a08-9f02-8a72188c6df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:36.951576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440af8a-f8f8-4a08-9f02-8a72188c6df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:36.951576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440af8a-f8f8-4a08-9f02-8a72188c6df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:36.951576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440af8a-f8f8-4a08-9f02-8a72188c6df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:36.953021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-4440af8a-f8f8-4a08-9f02-8a72188c6df0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:24:36.953021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Aug 24 11:24:36.953021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: Aug 24 11:24:36.959521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440af8a-f8f8-4a08-9f02-8a72188c6df0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.080275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 775/1546] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:36 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:24:37.084689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440af8a-f8f8-4a08-9f02-8a72188c6df0 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.122178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 772/1547] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:36 2022] POST /identity/v3/projects => generated 371 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:24:37.123478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccf2c1-6cde-48e1-beac-ff1840f0db00 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.185500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440af8a-f8f8-4a08-9f02-8a72188c6df0 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.279060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2576d-ce38-4a14-b086-0949369cc54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.279645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2576d-ce38-4a14-b086-0949369cc54d admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.279939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2576d-ce38-4a14-b086-0949369cc54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.280244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2576d-ce38-4a14-b086-0949369cc54d admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.287773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2576d-ce38-4a14-b086-0949369cc54d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.289695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2576d-ce38-4a14-b086-0949369cc54d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.324333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccf2c1-6cde-48e1-beac-ff1840f0db00 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.339529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cecf2-aa0d-4146-9432-07ce9f6931f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.340071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cecf2-aa0d-4146-9432-07ce9f6931f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.340353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cecf2-aa0d-4146-9432-07ce9f6931f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.340626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cecf2-aa0d-4146-9432-07ce9f6931f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.341260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cecf2-aa0d-4146-9432-07ce9f6931f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.342306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-991cecf2-aa0d-4146-9432-07ce9f6931f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:24:37.342306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Aug 24 11:24:37.342306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: Aug 24 11:24:37.344074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cecf2-aa0d-4146-9432-07ce9f6931f2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.424974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 776/1548] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/users => generated 286 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:37.426415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873eba5-ef85-460a-8e24-03b9b27c58b0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.426750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873eba5-ef85-460a-8e24-03b9b27c58b0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.427019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873eba5-ef85-460a-8e24-03b9b27c58b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.455569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4873eba5-ef85-460a-8e24-03b9b27c58b0 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.457092 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4873eba5-ef85-460a-8e24-03b9b27c58b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.461414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 773/1549] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/users => generated 286 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:37.462720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4e615-5b7a-4b8c-9b88-045d82da8ce3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.462999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4e615-5b7a-4b8c-9b88-045d82da8ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.463279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4e615-5b7a-4b8c-9b88-045d82da8ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.467082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 777/1550] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.468614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873eba5-ef85-460a-8e24-03b9b27c58b0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.479753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873eba5-ef85-460a-8e24-03b9b27c58b0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.484837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889185d4-4add-46d1-95f5-42461839b456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.486104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889185d4-4add-46d1-95f5-42461839b456 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.486104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889185d4-4add-46d1-95f5-42461839b456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.486340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889185d4-4add-46d1-95f5-42461839b456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.486861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889185d4-4add-46d1-95f5-42461839b456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.488498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889185d4-4add-46d1-95f5-42461839b456 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.502025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 778/1551] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:37.503330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cda88-6876-4cdc-87cc-79afbb23f3b5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.503620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cda88-6876-4cdc-87cc-79afbb23f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.503900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cda88-6876-4cdc-87cc-79afbb23f3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.504562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b7e4e615-5b7a-4b8c-9b88-045d82da8ce3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.506131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b7e4e615-5b7a-4b8c-9b88-045d82da8ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.516523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 774/1552] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.517367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4e615-5b7a-4b8c-9b88-045d82da8ce3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.526023 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c42cda88-6876-4cdc-87cc-79afbb23f3b5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.527320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c42cda88-6876-4cdc-87cc-79afbb23f3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.527443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4e615-5b7a-4b8c-9b88-045d82da8ce3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.532144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728119a5-faf6-4072-9067-fe7828aeac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.549067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728119a5-faf6-4072-9067-fe7828aeac67 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.549067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728119a5-faf6-4072-9067-fe7828aeac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.549067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728119a5-faf6-4072-9067-fe7828aeac67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.549067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728119a5-faf6-4072-9067-fe7828aeac67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.549067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728119a5-faf6-4072-9067-fe7828aeac67 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.549564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 779/1553] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.549564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3090d93-badd-4d04-98fa-f484351d34da None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.549564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3090d93-badd-4d04-98fa-f484351d34da None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.549564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3090d93-badd-4d04-98fa-f484351d34da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.549934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 775/1554] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:37.549934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728119a5-faf6-4072-9067-fe7828aeac67 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.560899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728119a5-faf6-4072-9067-fe7828aeac67 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.562562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c3090d93-badd-4d04-98fa-f484351d34da None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.563790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c3090d93-badd-4d04-98fa-f484351d34da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.566222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec0715-67d4-41db-b444-237d041406f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.566829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec0715-67d4-41db-b444-237d041406f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.567109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec0715-67d4-41db-b444-237d041406f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.567385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec0715-67d4-41db-b444-237d041406f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.568032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec0715-67d4-41db-b444-237d041406f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.569626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec0715-67d4-41db-b444-237d041406f3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.572302 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 780/1555] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.573990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1ece-f910-4d7d-9e23-d42902d3f01a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.574279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1ece-f910-4d7d-9e23-d42902d3f01a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.574556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1ece-f910-4d7d-9e23-d42902d3f01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.581943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 776/1556] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:37.581943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eda71e-88dc-474d-8375-43e358f8bd1e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.582242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eda71e-88dc-474d-8375-43e358f8bd1e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.582349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eda71e-88dc-474d-8375-43e358f8bd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.594585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-64af1ece-f910-4d7d-9e23-d42902d3f01a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.595731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-64af1ece-f910-4d7d-9e23-d42902d3f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.601410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b3eda71e-88dc-474d-8375-43e358f8bd1e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.602631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b3eda71e-88dc-474d-8375-43e358f8bd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.603964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 781/1557] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.605863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0d8ab-a785-4983-ab77-db9f4c0fa127 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.606175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0d8ab-a785-4983-ab77-db9f4c0fa127 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.606456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0d8ab-a785-4983-ab77-db9f4c0fa127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.611924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 777/1558] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.612792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eda71e-88dc-474d-8375-43e358f8bd1e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.618019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:37.623552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eda71e-88dc-474d-8375-43e358f8bd1e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.636193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ad723-a992-4980-8022-3f3b82a3f473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.636193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ad723-a992-4980-8022-3f3b82a3f473 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.636193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ad723-a992-4980-8022-3f3b82a3f473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.636193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ad723-a992-4980-8022-3f3b82a3f473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.636550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0fe0d8ab-a785-4983-ab77-db9f4c0fa127 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.636658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ad723-a992-4980-8022-3f3b82a3f473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.636756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0fe0d8ab-a785-4983-ab77-db9f4c0fa127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.636845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ad723-a992-4980-8022-3f3b82a3f473 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.640766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 782/1559] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.644964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ad974-bc04-4f55-b17a-44edf3697b11 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.644964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ad974-bc04-4f55-b17a-44edf3697b11 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.644964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ad974-bc04-4f55-b17a-44edf3697b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.646316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 778/1560] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:37.647647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9137010-8110-4b3e-9811-babd96aa35e3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.647891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9137010-8110-4b3e-9811-babd96aa35e3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.648308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9137010-8110-4b3e-9811-babd96aa35e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.662979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-840ad974-bc04-4f55-b17a-44edf3697b11 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.664181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-840ad974-bc04-4f55-b17a-44edf3697b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.667540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c9137010-8110-4b3e-9811-babd96aa35e3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.671487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c9137010-8110-4b3e-9811-babd96aa35e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.673024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 783/1561] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.673811 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ad974-bc04-4f55-b17a-44edf3697b11 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.678096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 779/1562] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.678877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9137010-8110-4b3e-9811-babd96aa35e3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.689252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9137010-8110-4b3e-9811-babd96aa35e3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.689531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ad974-bc04-4f55-b17a-44edf3697b11 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.694056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e7b08-c7fb-42f0-87db-d8eadc829840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.694703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e7b08-c7fb-42f0-87db-d8eadc829840 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.694978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e7b08-c7fb-42f0-87db-d8eadc829840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.695074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acc4ef-07d3-42f8-a0c7-44c1e5ed3eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.695248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e7b08-c7fb-42f0-87db-d8eadc829840 admin admin] PATH_INFO: `/v3/projects/4261e17a79fa4878a2fd3da019e13b5b/users/5baa38db92664d50a96672b9f57f4c3b/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.695694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acc4ef-07d3-42f8-a0c7-44c1e5ed3eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.695981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acc4ef-07d3-42f8-a0c7-44c1e5ed3eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.696334 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acc4ef-07d3-42f8-a0c7-44c1e5ed3eeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.696999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acc4ef-07d3-42f8-a0c7-44c1e5ed3eeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:37.697406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acc4ef-07d3-42f8-a0c7-44c1e5ed3eeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.699061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acc4ef-07d3-42f8-a0c7-44c1e5ed3eeb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.710851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e7b08-c7fb-42f0-87db-d8eadc829840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4261e17a79fa4878a2fd3da019e13b5b, user_id=5baa38db92664d50a96672b9f57f4c3b, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.711485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 784/1563] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:37.712712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb52a9-a4dd-4a58-b51e-ec811cf79ab9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.713308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb52a9-a4dd-4a58-b51e-ec811cf79ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.713308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb52a9-a4dd-4a58-b51e-ec811cf79ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.713624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e7b08-c7fb-42f0-87db-d8eadc829840 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.736137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1feb52a9-a4dd-4a58-b51e-ec811cf79ab9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.737376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1feb52a9-a4dd-4a58-b51e-ec811cf79ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.745480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 785/1564] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.747932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca808976-d62f-427c-8cfb-b57aa7e04b43 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.748205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca808976-d62f-427c-8cfb-b57aa7e04b43 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.748472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca808976-d62f-427c-8cfb-b57aa7e04b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.768129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ca808976-d62f-427c-8cfb-b57aa7e04b43 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.769429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ca808976-d62f-427c-8cfb-b57aa7e04b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.777156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 786/1565] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.778091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca808976-d62f-427c-8cfb-b57aa7e04b43 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.788232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca808976-d62f-427c-8cfb-b57aa7e04b43 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.793180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bc281-27f0-4782-b5d4-cda2ec8f7275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.793761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bc281-27f0-4782-b5d4-cda2ec8f7275 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.794075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bc281-27f0-4782-b5d4-cda2ec8f7275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.794348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bc281-27f0-4782-b5d4-cda2ec8f7275 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.794989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bc281-27f0-4782-b5d4-cda2ec8f7275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:37.795296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bc281-27f0-4782-b5d4-cda2ec8f7275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.796908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bc281-27f0-4782-b5d4-cda2ec8f7275 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.808685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 787/1566] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:37.809507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bc281-27f0-4782-b5d4-cda2ec8f7275 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.817515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 780/1567] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:37 2022] PUT /identity/v3/projects/4261e17a79fa4878a2fd3da019e13b5b/users/5baa38db92664d50a96672b9f57f4c3b/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:37.818394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e7b08-c7fb-42f0-87db-d8eadc829840 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.821666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bc281-27f0-4782-b5d4-cda2ec8f7275 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.844467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:37.880636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e7b08-c7fb-42f0-87db-d8eadc829840 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.882786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da278640-794c-4863-9515-08094bf1c036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.883508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da278640-794c-4863-9515-08094bf1c036 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.883752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da278640-794c-4863-9515-08094bf1c036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.884070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da278640-794c-4863-9515-08094bf1c036 admin admin] PATH_INFO: `/v3/projects/7f363d569e794bdc94d4ec77f637dd5e/users/1f35fad1139349ca94e6b7356c01f099/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.885819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fe16c-8918-4e3b-b2fd-d83aedd0025d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.887005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fe16c-8918-4e3b-b2fd-d83aedd0025d admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.887432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fe16c-8918-4e3b-b2fd-d83aedd0025d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.887768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fe16c-8918-4e3b-b2fd-d83aedd0025d admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.888425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fe16c-8918-4e3b-b2fd-d83aedd0025d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:37.888775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fe16c-8918-4e3b-b2fd-d83aedd0025d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.891044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fe16c-8918-4e3b-b2fd-d83aedd0025d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.899982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da278640-794c-4863-9515-08094bf1c036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f363d569e794bdc94d4ec77f637dd5e, user_id=1f35fad1139349ca94e6b7356c01f099, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.902001 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-da278640-794c-4863-9515-08094bf1c036 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:24:37.902001 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Aug 24 11:24:37.902001 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: Aug 24 11:24:37.903216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da278640-794c-4863-9515-08094bf1c036 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.903360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 781/1568] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:37.904523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a94c-5c74-42de-88d4-679cf41df21e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.904814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a94c-5c74-42de-88d4-679cf41df21e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.905091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a94c-5c74-42de-88d4-679cf41df21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.925772 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1189a94c-5c74-42de-88d4-679cf41df21e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.927026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1189a94c-5c74-42de-88d4-679cf41df21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.934938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 782/1569] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.935813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189a94c-5c74-42de-88d4-679cf41df21e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.945407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189a94c-5c74-42de-88d4-679cf41df21e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.949965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f3acc-39bd-4f3f-b9b2-70a6b098e463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.952900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f3acc-39bd-4f3f-b9b2-70a6b098e463 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.952900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f3acc-39bd-4f3f-b9b2-70a6b098e463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.952900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f3acc-39bd-4f3f-b9b2-70a6b098e463 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.952900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f3acc-39bd-4f3f-b9b2-70a6b098e463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.954174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f3acc-39bd-4f3f-b9b2-70a6b098e463 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.977359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 788/1570] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:37 2022] PUT /identity/v3/projects/7f363d569e794bdc94d4ec77f637dd5e/users/1f35fad1139349ca94e6b7356c01f099/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:37.978231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da278640-794c-4863-9515-08094bf1c036 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.036215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 783/1571] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:38.037145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f3acc-39bd-4f3f-b9b2-70a6b098e463 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.042005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da278640-794c-4863-9515-08094bf1c036 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.084091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:38.101240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002197be-16c7-4ccf-ac77-eb68f9218e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.101580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f3acc-39bd-4f3f-b9b2-70a6b098e463 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.101924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002197be-16c7-4ccf-ac77-eb68f9218e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.102192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002197be-16c7-4ccf-ac77-eb68f9218e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.102475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002197be-16c7-4ccf-ac77-eb68f9218e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.103129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002197be-16c7-4ccf-ac77-eb68f9218e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:38.103442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002197be-16c7-4ccf-ac77-eb68f9218e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.105059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002197be-16c7-4ccf-ac77-eb68f9218e4b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.106732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea7120-48a3-4c39-bae3-7b324491d971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.107338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea7120-48a3-4c39-bae3-7b324491d971 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.107623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea7120-48a3-4c39-bae3-7b324491d971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.107899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea7120-48a3-4c39-bae3-7b324491d971 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.108632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea7120-48a3-4c39-bae3-7b324491d971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:38.108942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea7120-48a3-4c39-bae3-7b324491d971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.111039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea7120-48a3-4c39-bae3-7b324491d971 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.117449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 789/1572] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:38.127365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002197be-16c7-4ccf-ac77-eb68f9218e4b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.127646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 784/1573] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:38.127646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe772-b2e1-464a-b4cb-fe458dedb232 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.127646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe772-b2e1-464a-b4cb-fe458dedb232 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.127646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe772-b2e1-464a-b4cb-fe458dedb232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.132508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002197be-16c7-4ccf-ac77-eb68f9218e4b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.141204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ebe00-0066-41d4-9408-33e3e0f85f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.141822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ebe00-0066-41d4-9408-33e3e0f85f68 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.142134 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ebe00-0066-41d4-9408-33e3e0f85f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.142429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ebe00-0066-41d4-9408-33e3e0f85f68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.143049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ebe00-0066-41d4-9408-33e3e0f85f68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.145178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ebe00-0066-41d4-9408-33e3e0f85f68 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.155787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3babe772-b2e1-464a-b4cb-fe458dedb232 None None] MFA Rules not processed for user `5baa38db92664d50a96672b9f57f4c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:38.157468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3babe772-b2e1-464a-b4cb-fe458dedb232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:38.221474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 785/1574] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:38.226800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe772-b2e1-464a-b4cb-fe458dedb232 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.229040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 790/1575] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:38.229775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ebe00-0066-41d4-9408-33e3e0f85f68 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.240386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe772-b2e1-464a-b4cb-fe458dedb232 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.250569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:38.297138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ebe00-0066-41d4-9408-33e3e0f85f68 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.302037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82febaf-b735-441b-a326-16aa1599a0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.302648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82febaf-b735-441b-a326-16aa1599a0ff admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.302929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82febaf-b735-441b-a326-16aa1599a0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.303200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82febaf-b735-441b-a326-16aa1599a0ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.303859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82febaf-b735-441b-a326-16aa1599a0ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:38.304163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82febaf-b735-441b-a326-16aa1599a0ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.305805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82febaf-b735-441b-a326-16aa1599a0ff admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.306283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143c118-cb86-4c6d-abf5-3a83940eb096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.307026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143c118-cb86-4c6d-abf5-3a83940eb096 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.307320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143c118-cb86-4c6d-abf5-3a83940eb096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.307656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143c118-cb86-4c6d-abf5-3a83940eb096 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.308291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143c118-cb86-4c6d-abf5-3a83940eb096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.310318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143c118-cb86-4c6d-abf5-3a83940eb096 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.319301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 791/1576] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:38.322257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fc788-c741-49c7-83ae-0f1536df4255 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.322257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fc788-c741-49c7-83ae-0f1536df4255 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.322257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fc788-c741-49c7-83ae-0f1536df4255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.348888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-182fc788-c741-49c7-83ae-0f1536df4255 None None] MFA Rules not processed for user `1f35fad1139349ca94e6b7356c01f099`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:38.350969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-182fc788-c741-49c7-83ae-0f1536df4255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:38.414028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 786/1577] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/projects => generated 367 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:38.415999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 792/1578] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:38.416138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143c118-cb86-4c6d-abf5-3a83940eb096 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.416224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fc788-c741-49c7-83ae-0f1536df4255 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.427508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fc788-c741-49c7-83ae-0f1536df4255 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.468186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:38.482424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143c118-cb86-4c6d-abf5-3a83940eb096 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.487663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639b9b0-a9a3-400d-b671-d267622d6524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.488276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639b9b0-a9a3-400d-b671-d267622d6524 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.488557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639b9b0-a9a3-400d-b671-d267622d6524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.488834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639b9b0-a9a3-400d-b671-d267622d6524 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.489466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639b9b0-a9a3-400d-b671-d267622d6524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.491439 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639b9b0-a9a3-400d-b671-d267622d6524 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.492144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511869ed-73f1-43aa-871e-5a07da6d8349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.492745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511869ed-73f1-43aa-871e-5a07da6d8349 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.493024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511869ed-73f1-43aa-871e-5a07da6d8349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.493294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511869ed-73f1-43aa-871e-5a07da6d8349 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.493962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511869ed-73f1-43aa-871e-5a07da6d8349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.495907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511869ed-73f1-43aa-871e-5a07da6d8349 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.600954 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 787/1579] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:38.601893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639b9b0-a9a3-400d-b671-d267622d6524 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.613097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639b9b0-a9a3-400d-b671-d267622d6524 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.620062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634394a-e592-402d-9c65-c211ae161001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.620062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634394a-e592-402d-9c65-c211ae161001 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.620062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634394a-e592-402d-9c65-c211ae161001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.620062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634394a-e592-402d-9c65-c211ae161001 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.620559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634394a-e592-402d-9c65-c211ae161001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.620869 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 793/1580] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/projects => generated 385 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:38.621699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511869ed-73f1-43aa-871e-5a07da6d8349 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.622287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634394a-e592-402d-9c65-c211ae161001 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.686938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511869ed-73f1-43aa-871e-5a07da6d8349 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.691502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043cf54-b19b-4484-af0d-96adf95c02e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.694266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043cf54-b19b-4484-af0d-96adf95c02e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.694266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043cf54-b19b-4484-af0d-96adf95c02e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.694266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043cf54-b19b-4484-af0d-96adf95c02e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.694266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043cf54-b19b-4484-af0d-96adf95c02e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.695146 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043cf54-b19b-4484-af0d-96adf95c02e6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.720863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 788/1581] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:38.722181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fda31-aa7b-4e0e-9241-15b6b5634ae4 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.722596 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fda31-aa7b-4e0e-9241-15b6b5634ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.722874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fda31-aa7b-4e0e-9241-15b6b5634ae4 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.724306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 789/1582] 10.222.0.83 () {54 vars in 962 bytes} [Wed Aug 24 11:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:38.724801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fda31-aa7b-4e0e-9241-15b6b5634ae4 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.788916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fda31-aa7b-4e0e-9241-15b6b5634ae4 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.794033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fab9da-1386-4bd7-bab3-1ffadc3daec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.794598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 794/1583] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:38.795054 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fab9da-1386-4bd7-bab3-1ffadc3daec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.795054 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fab9da-1386-4bd7-bab3-1ffadc3daec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.795257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fab9da-1386-4bd7-bab3-1ffadc3daec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.795483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043cf54-b19b-4484-af0d-96adf95c02e6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.796470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fab9da-1386-4bd7-bab3-1ffadc3daec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.798559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fab9da-1386-4bd7-bab3-1ffadc3daec1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.806192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043cf54-b19b-4484-af0d-96adf95c02e6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.809209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:38.811028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebb67b-9c0c-422a-92a6-b11d9ecb078b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.811642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebb67b-9c0c-422a-92a6-b11d9ecb078b admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.811901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebb67b-9c0c-422a-92a6-b11d9ecb078b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.812170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebb67b-9c0c-422a-92a6-b11d9ecb078b admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.812792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebb67b-9c0c-422a-92a6-b11d9ecb078b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.814746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebb67b-9c0c-422a-92a6-b11d9ecb078b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.875993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 790/1584] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:38.876908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fab9da-1386-4bd7-bab3-1ffadc3daec1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.897209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 795/1585] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:38.898142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebb67b-9c0c-422a-92a6-b11d9ecb078b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.908384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebb67b-9c0c-422a-92a6-b11d9ecb078b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.944412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fab9da-1386-4bd7-bab3-1ffadc3daec1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:38.948932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5804b-8858-4528-b029-c4727dcb656c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.949534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5804b-8858-4528-b029-c4727dcb656c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.949810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5804b-8858-4528-b029-c4727dcb656c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.950114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5804b-8858-4528-b029-c4727dcb656c admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.950736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5804b-8858-4528-b029-c4727dcb656c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.952712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5804b-8858-4528-b029-c4727dcb656c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.967680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71dd800-9d5d-444b-9149-676276b30c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:38.968820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dd800-9d5d-444b-9149-676276b30c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:38.969103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dd800-9d5d-444b-9149-676276b30c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:38.969374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dd800-9d5d-444b-9149-676276b30c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:38.970162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71dd800-9d5d-444b-9149-676276b30c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:38.971817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71dd800-9d5d-444b-9149-676276b30c9e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:38.982843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 796/1586] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:38.984341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71dd800-9d5d-444b-9149-676276b30c9e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:38.995393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71dd800-9d5d-444b-9149-676276b30c9e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.000337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486337a-4cec-421e-b737-b798264a36dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.001566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486337a-4cec-421e-b737-b798264a36dd admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.001566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486337a-4cec-421e-b737-b798264a36dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.001566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486337a-4cec-421e-b737-b798264a36dd admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.002180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486337a-4cec-421e-b737-b798264a36dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.004187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486337a-4cec-421e-b737-b798264a36dd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.030289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 791/1587] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:39.031694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5804b-8858-4528-b029-c4727dcb656c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.095933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5804b-8858-4528-b029-c4727dcb656c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.100585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1242a-ba87-45bb-a757-a4a323f6a02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.101184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1242a-ba87-45bb-a757-a4a323f6a02d admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.101464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1242a-ba87-45bb-a757-a4a323f6a02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.101737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1242a-ba87-45bb-a757-a4a323f6a02d admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.102396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1242a-ba87-45bb-a757-a4a323f6a02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.104320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1242a-ba87-45bb-a757-a4a323f6a02d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.114852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 797/1588] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:38 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:39.116311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e2349-8b6e-4e91-91b6-b953377bb23f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.116599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e2349-8b6e-4e91-91b6-b953377bb23f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.116878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e2349-8b6e-4e91-91b6-b953377bb23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.142494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:39.157013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0d8e2349-8b6e-4e91-91b6-b953377bb23f None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:39.158272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0d8e2349-8b6e-4e91-91b6-b953377bb23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:39.199824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 792/1589] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:39 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:39.199824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1242a-ba87-45bb-a757-a4a323f6a02d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.207828 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1242a-ba87-45bb-a757-a4a323f6a02d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.212689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1840a1d-ce84-4fe2-b070-9f9cd0cabc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.213292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1840a1d-ce84-4fe2-b070-9f9cd0cabc6b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.213575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1840a1d-ce84-4fe2-b070-9f9cd0cabc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.213847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1840a1d-ce84-4fe2-b070-9f9cd0cabc6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.214530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1840a1d-ce84-4fe2-b070-9f9cd0cabc6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.216142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1840a1d-ce84-4fe2-b070-9f9cd0cabc6b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.227363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 793/1590] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.228260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1840a1d-ce84-4fe2-b070-9f9cd0cabc6b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.232077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 798/1591] 10.222.0.83 () {58 vars in 1059 bytes} [Wed Aug 24 11:24:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:39.236920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e2349-8b6e-4e91-91b6-b953377bb23f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.238591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1840a1d-ce84-4fe2-b070-9f9cd0cabc6b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.243677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847997b-dceb-4df9-888e-f418529ac700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.244279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847997b-dceb-4df9-888e-f418529ac700 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.244551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847997b-dceb-4df9-888e-f418529ac700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.244825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847997b-dceb-4df9-888e-f418529ac700 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.245450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847997b-dceb-4df9-888e-f418529ac700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.246249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e2349-8b6e-4e91-91b6-b953377bb23f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.247396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847997b-dceb-4df9-888e-f418529ac700 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.251636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b1b58-ca3a-4334-945e-0f20e46be169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.252372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b1b58-ca3a-4334-945e-0f20e46be169 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.252715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b1b58-ca3a-4334-945e-0f20e46be169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.253060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b1b58-ca3a-4334-945e-0f20e46be169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.253838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b1b58-ca3a-4334-945e-0f20e46be169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.256238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b1b58-ca3a-4334-945e-0f20e46be169 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.268601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 799/1592] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.271213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b1b58-ca3a-4334-945e-0f20e46be169 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.279056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b1b58-ca3a-4334-945e-0f20e46be169 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.284230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30d605-de9d-4dc7-8d87-754e8290d14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.284812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30d605-de9d-4dc7-8d87-754e8290d14c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.285087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30d605-de9d-4dc7-8d87-754e8290d14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.285360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30d605-de9d-4dc7-8d87-754e8290d14c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.286034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30d605-de9d-4dc7-8d87-754e8290d14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.287619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30d605-de9d-4dc7-8d87-754e8290d14c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.297951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 800/1593] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.298719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30d605-de9d-4dc7-8d87-754e8290d14c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.307975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30d605-de9d-4dc7-8d87-754e8290d14c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.312583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d6445-3e71-4f97-9c23-31714aac2544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.313153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d6445-3e71-4f97-9c23-31714aac2544 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.313428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d6445-3e71-4f97-9c23-31714aac2544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.313699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d6445-3e71-4f97-9c23-31714aac2544 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.314343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d6445-3e71-4f97-9c23-31714aac2544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.316243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d6445-3e71-4f97-9c23-31714aac2544 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.336028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 794/1594] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:39 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:39.340060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847997b-dceb-4df9-888e-f418529ac700 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.347021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847997b-dceb-4df9-888e-f418529ac700 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.351677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb900c-32c0-4d3a-a999-d8f85b19d0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.352315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb900c-32c0-4d3a-a999-d8f85b19d0da admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.352598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb900c-32c0-4d3a-a999-d8f85b19d0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.352873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb900c-32c0-4d3a-a999-d8f85b19d0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.353511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb900c-32c0-4d3a-a999-d8f85b19d0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.355236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb900c-32c0-4d3a-a999-d8f85b19d0da admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.365359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 795/1595] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.366235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb900c-32c0-4d3a-a999-d8f85b19d0da admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.376822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb900c-32c0-4d3a-a999-d8f85b19d0da admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.381696 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025ac75-15fb-4f5a-be69-1456a03a2ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.382333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025ac75-15fb-4f5a-be69-1456a03a2ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.382616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025ac75-15fb-4f5a-be69-1456a03a2ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.382890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025ac75-15fb-4f5a-be69-1456a03a2ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.383541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025ac75-15fb-4f5a-be69-1456a03a2ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.385148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025ac75-15fb-4f5a-be69-1456a03a2ea8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.396442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 796/1596] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.397302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025ac75-15fb-4f5a-be69-1456a03a2ea8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.407069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 801/1597] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:39 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:39.407290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025ac75-15fb-4f5a-be69-1456a03a2ea8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.407996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d6445-3e71-4f97-9c23-31714aac2544 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.414593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee19a3-451b-49f2-981d-1276b115c496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.418021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee19a3-451b-49f2-981d-1276b115c496 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.418021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee19a3-451b-49f2-981d-1276b115c496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.418021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee19a3-451b-49f2-981d-1276b115c496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.418021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee19a3-451b-49f2-981d-1276b115c496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.418447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee19a3-451b-49f2-981d-1276b115c496 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.421293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d6445-3e71-4f97-9c23-31714aac2544 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.427448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b4e8f-f374-4b6e-a13d-ce52ee7f2f16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.428062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b4e8f-f374-4b6e-a13d-ce52ee7f2f16 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.428333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b4e8f-f374-4b6e-a13d-ce52ee7f2f16 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.428604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b4e8f-f374-4b6e-a13d-ce52ee7f2f16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.429530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 797/1598] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.429530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee19a3-451b-49f2-981d-1276b115c496 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.435825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:39.442964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee19a3-451b-49f2-981d-1276b115c496 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.448373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9de3e5-e5ee-43d0-90dc-7ac270e20574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.448974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9de3e5-e5ee-43d0-90dc-7ac270e20574 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.449253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9de3e5-e5ee-43d0-90dc-7ac270e20574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.449524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9de3e5-e5ee-43d0-90dc-7ac270e20574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.450316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9de3e5-e5ee-43d0-90dc-7ac270e20574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.451879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9de3e5-e5ee-43d0-90dc-7ac270e20574 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.462992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 798/1599] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.463927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9de3e5-e5ee-43d0-90dc-7ac270e20574 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.474662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9de3e5-e5ee-43d0-90dc-7ac270e20574 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.480097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750aac8e-f313-4d32-967b-99de78b02216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.480785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750aac8e-f313-4d32-967b-99de78b02216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.481069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750aac8e-f313-4d32-967b-99de78b02216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.481374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750aac8e-f313-4d32-967b-99de78b02216 admin admin] PATH_INFO: `/v3/projects/86e8de8aeabf4b8fa67ed0784fc39e46/users/cd79bc9c3a694fe2a4e3c645e682e438/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.488551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b4e8f-f374-4b6e-a13d-ce52ee7f2f16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.490200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b4e8f-f374-4b6e-a13d-ce52ee7f2f16 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.496872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750aac8e-f313-4d32-967b-99de78b02216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e8de8aeabf4b8fa67ed0784fc39e46, user_id=cd79bc9c3a694fe2a4e3c645e682e438, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.500453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 802/1600] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:39.500603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750aac8e-f313-4d32-967b-99de78b02216 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.501123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b4e8f-f374-4b6e-a13d-ce52ee7f2f16 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.511129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b4e8f-f374-4b6e-a13d-ce52ee7f2f16 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.516349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501063e3-2e90-4b30-8161-0dfdc61ef389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.517063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501063e3-2e90-4b30-8161-0dfdc61ef389 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.517354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501063e3-2e90-4b30-8161-0dfdc61ef389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.517623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501063e3-2e90-4b30-8161-0dfdc61ef389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.518371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501063e3-2e90-4b30-8161-0dfdc61ef389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.520104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501063e3-2e90-4b30-8161-0dfdc61ef389 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.530820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 803/1601] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.531901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501063e3-2e90-4b30-8161-0dfdc61ef389 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.542538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501063e3-2e90-4b30-8161-0dfdc61ef389 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.547498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475c233-f6ea-483d-a0ee-89cae1d4bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.549627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475c233-f6ea-483d-a0ee-89cae1d4bd9b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.549627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475c233-f6ea-483d-a0ee-89cae1d4bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.549627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475c233-f6ea-483d-a0ee-89cae1d4bd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.549627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475c233-f6ea-483d-a0ee-89cae1d4bd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.550994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475c233-f6ea-483d-a0ee-89cae1d4bd9b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.561343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 804/1602] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.562689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475c233-f6ea-483d-a0ee-89cae1d4bd9b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.573047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475c233-f6ea-483d-a0ee-89cae1d4bd9b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.576510 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 799/1603] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:39 2022] PUT /identity/v3/projects/86e8de8aeabf4b8fa67ed0784fc39e46/users/cd79bc9c3a694fe2a4e3c645e682e438/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:39.581611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750aac8e-f313-4d32-967b-99de78b02216 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.600582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:39.645578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9f21c-069e-43c8-a9ee-d2feb46749b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.646802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9f21c-069e-43c8-a9ee-d2feb46749b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.647211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9f21c-069e-43c8-a9ee-d2feb46749b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.648050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9f21c-069e-43c8-a9ee-d2feb46749b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.648716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9f21c-069e-43c8-a9ee-d2feb46749b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.650546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9f21c-069e-43c8-a9ee-d2feb46749b2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.651342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750aac8e-f313-4d32-967b-99de78b02216 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.656793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f754f10-6c51-41de-b302-fadf53f32db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.657139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f754f10-6c51-41de-b302-fadf53f32db8 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.657490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f754f10-6c51-41de-b302-fadf53f32db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.657854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f754f10-6c51-41de-b302-fadf53f32db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.658594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f754f10-6c51-41de-b302-fadf53f32db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.660258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f754f10-6c51-41de-b302-fadf53f32db8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.665262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 805/1604] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.666554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9f21c-069e-43c8-a9ee-d2feb46749b2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.671870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 800/1605] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.675827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f754f10-6c51-41de-b302-fadf53f32db8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.676582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9f21c-069e-43c8-a9ee-d2feb46749b2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.681933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088a88-a230-48e3-9077-d143a9266a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.682877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f754f10-6c51-41de-b302-fadf53f32db8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.683379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088a88-a230-48e3-9077-d143a9266a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.683985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088a88-a230-48e3-9077-d143a9266a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.684476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088a88-a230-48e3-9077-d143a9266a56 admin admin] PATH_INFO: `/v3/projects/b4b6d46119374e6ab76032a4db1198d5/users/4aa39f218d4d463291ffe15b929c4e2d/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.688714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377af0c-e244-4f08-9357-5d148f8e6732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.689388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377af0c-e244-4f08-9357-5d148f8e6732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.689647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377af0c-e244-4f08-9357-5d148f8e6732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.689949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377af0c-e244-4f08-9357-5d148f8e6732 admin admin] PATH_INFO: `/v3/projects/6cb495e455b848aba1d66a45c85f1f61/users/7cf99bf2d43b4b0a8d2cb48aead512bf/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.705159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088a88-a230-48e3-9077-d143a9266a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b6d46119374e6ab76032a4db1198d5, user_id=4aa39f218d4d463291ffe15b929c4e2d, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.705335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377af0c-e244-4f08-9357-5d148f8e6732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb495e455b848aba1d66a45c85f1f61, user_id=7cf99bf2d43b4b0a8d2cb48aead512bf, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.707946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377af0c-e244-4f08-9357-5d148f8e6732 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.713209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088a88-a230-48e3-9077-d143a9266a56 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.812276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 801/1606] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:39 2022] PUT /identity/v3/projects/6cb495e455b848aba1d66a45c85f1f61/users/7cf99bf2d43b4b0a8d2cb48aead512bf/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:39.813096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377af0c-e244-4f08-9357-5d148f8e6732 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.834614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 806/1607] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:39 2022] PUT /identity/v3/projects/b4b6d46119374e6ab76032a4db1198d5/users/4aa39f218d4d463291ffe15b929c4e2d/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:39.835420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088a88-a230-48e3-9077-d143a9266a56 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.875659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377af0c-e244-4f08-9357-5d148f8e6732 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.904164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088a88-a230-48e3-9077-d143a9266a56 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.908865 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1da72-a695-4288-baa2-ed0e37654c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.909518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1da72-a695-4288-baa2-ed0e37654c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.909802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1da72-a695-4288-baa2-ed0e37654c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.910103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1da72-a695-4288-baa2-ed0e37654c70 admin admin] PATH_INFO: `/v3/projects/6e41187605624091a4edc1993c4357e4/users/92712c36310c440c94da6c432c77a895/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.919129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:39.925940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1da72-a695-4288-baa2-ed0e37654c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e41187605624091a4edc1993c4357e4, user_id=92712c36310c440c94da6c432c77a895, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.928601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1da72-a695-4288-baa2-ed0e37654c70 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.932661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a96fd-d2ff-498f-abac-bd56d9d8b51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.936738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a96fd-d2ff-498f-abac-bd56d9d8b51e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.936738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a96fd-d2ff-498f-abac-bd56d9d8b51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.936738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a96fd-d2ff-498f-abac-bd56d9d8b51e admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.936738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a96fd-d2ff-498f-abac-bd56d9d8b51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:39.936738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a96fd-d2ff-498f-abac-bd56d9d8b51e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:39.946243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 802/1608] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:39.947424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22853-2797-4a7c-a45d-bf4a403b2014 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.947703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22853-2797-4a7c-a45d-bf4a403b2014 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.947975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22853-2797-4a7c-a45d-bf4a403b2014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:39.967650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-24f22853-2797-4a7c-a45d-bf4a403b2014 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:39.969237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-24f22853-2797-4a7c-a45d-bf4a403b2014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:39.977030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 803/1609] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:39.977760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f22853-2797-4a7c-a45d-bf4a403b2014 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:39.988714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f22853-2797-4a7c-a45d-bf4a403b2014 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:39.993371 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893b521-8d2f-4478-84d3-2befa45da70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:39.994063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893b521-8d2f-4478-84d3-2befa45da70c admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:39.994340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893b521-8d2f-4478-84d3-2befa45da70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:39.994608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893b521-8d2f-4478-84d3-2befa45da70c admin admin] PATH_INFO: `/v3/projects/b335b0177512496c9e6c1aa4b4f8532f/users/7f1a36036abc4e9b908a96b3107df5f9/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.009184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893b521-8d2f-4478-84d3-2befa45da70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b335b0177512496c9e6c1aa4b4f8532f, user_id=7f1a36036abc4e9b908a96b3107df5f9, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.011496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 807/1610] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:39 2022] PUT /identity/v3/projects/6e41187605624091a4edc1993c4357e4/users/92712c36310c440c94da6c432c77a895/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:40.012024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893b521-8d2f-4478-84d3-2befa45da70c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.012319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1da72-a695-4288-baa2-ed0e37654c70 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.074784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1da72-a695-4288-baa2-ed0e37654c70 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.080138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cfb55-ab40-4bca-84b1-39df3fd0ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.080806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cfb55-ab40-4bca-84b1-39df3fd0ce59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.081084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cfb55-ab40-4bca-84b1-39df3fd0ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.081358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cfb55-ab40-4bca-84b1-39df3fd0ce59 admin admin] PATH_INFO: `/v3/projects/c5ec4c189693458ab07bde1354c2a439/users/52d974fe2d8f46c0acaac2490ab833cf/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.086086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 804/1611] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:39 2022] PUT /identity/v3/projects/b335b0177512496c9e6c1aa4b4f8532f/users/7f1a36036abc4e9b908a96b3107df5f9/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:40.087469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81e2ef-ce93-4324-80b9-b7975f20db07 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.087835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81e2ef-ce93-4324-80b9-b7975f20db07 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.088114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81e2ef-ce93-4324-80b9-b7975f20db07 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.089468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 805/1612] 10.222.0.83 () {54 vars in 912 bytes} [Wed Aug 24 11:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:40.093373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dbd1f-1903-477d-aa89-98a9f4c2b5ce None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.093793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dbd1f-1903-477d-aa89-98a9f4c2b5ce None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.094094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dbd1f-1903-477d-aa89-98a9f4c2b5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.102002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cfb55-ab40-4bca-84b1-39df3fd0ce59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ec4c189693458ab07bde1354c2a439, user_id=52d974fe2d8f46c0acaac2490ab833cf, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.105654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cfb55-ab40-4bca-84b1-39df3fd0ce59 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.121669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-017dbd1f-1903-477d-aa89-98a9f4c2b5ce None None] MFA Rules not processed for user `7cf99bf2d43b4b0a8d2cb48aead512bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:40.123006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-017dbd1f-1903-477d-aa89-98a9f4c2b5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:40.188096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 806/1613] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:40.189533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e34080-98ad-4e33-8a63-e0311340e439 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.189818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e34080-98ad-4e33-8a63-e0311340e439 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.190130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e34080-98ad-4e33-8a63-e0311340e439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.204297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 808/1614] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:40 2022] PUT /identity/v3/projects/c5ec4c189693458ab07bde1354c2a439/users/52d974fe2d8f46c0acaac2490ab833cf/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:40.207633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cfb55-ab40-4bca-84b1-39df3fd0ce59 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.209074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-77e34080-98ad-4e33-8a63-e0311340e439 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:40.210325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-77e34080-98ad-4e33-8a63-e0311340e439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:40.214243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:40.269359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cfb55-ab40-4bca-84b1-39df3fd0ce59 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.272764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 807/1615] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:40.272764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e34080-98ad-4e33-8a63-e0311340e439 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.275984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4ef6a-ca33-43ea-8283-fa0d1e54d228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.276657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4ef6a-ca33-43ea-8283-fa0d1e54d228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.276936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4ef6a-ca33-43ea-8283-fa0d1e54d228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.277208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4ef6a-ca33-43ea-8283-fa0d1e54d228 admin admin] PATH_INFO: `/v3/projects/6546cd2088ee4c29890d6841ac1a2686/users/64db87b93e5544f1a6b8167e2003e200/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.297517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4ef6a-ca33-43ea-8283-fa0d1e54d228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6546cd2088ee4c29890d6841ac1a2686, user_id=64db87b93e5544f1a6b8167e2003e200, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.300256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4ef6a-ca33-43ea-8283-fa0d1e54d228 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.322523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:40.332664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e34080-98ad-4e33-8a63-e0311340e439 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.336293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7ed0a-4988-4d37-9eb3-013b7a0c9f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.336898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7ed0a-4988-4d37-9eb3-013b7a0c9f8b service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.337211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7ed0a-4988-4d37-9eb3-013b7a0c9f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.337487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7ed0a-4988-4d37-9eb3-013b7a0c9f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.346772 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7ed0a-4988-4d37-9eb3-013b7a0c9f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.348195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7ed0a-4988-4d37-9eb3-013b7a0c9f8b service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.357410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 808/1616] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:40.359296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a7815-36ec-4c73-9553-3d65dab4dd22 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.359582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a7815-36ec-4c73-9553-3d65dab4dd22 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.359858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a7815-36ec-4c73-9553-3d65dab4dd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.378289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5c5a7815-36ec-4c73-9553-3d65dab4dd22 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:40.379221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5c5a7815-36ec-4c73-9553-3d65dab4dd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:40.386996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 809/1617] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:40.392055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ec2d9-a92d-42d8-b42c-8f52423d5762 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.392055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ec2d9-a92d-42d8-b42c-8f52423d5762 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.392055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ec2d9-a92d-42d8-b42c-8f52423d5762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.394399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 809/1618] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:40 2022] PUT /identity/v3/projects/6546cd2088ee4c29890d6841ac1a2686/users/64db87b93e5544f1a6b8167e2003e200/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:40.396569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0a50-141a-468b-b41e-c47cdcb76e2a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.397060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0a50-141a-468b-b41e-c47cdcb76e2a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.397531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0a50-141a-468b-b41e-c47cdcb76e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.422099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b40ec2d9-a92d-42d8-b42c-8f52423d5762 None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:40.423332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b40ec2d9-a92d-42d8-b42c-8f52423d5762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:40.435928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ff2e0a50-141a-468b-b41e-c47cdcb76e2a None None] MFA Rules not processed for user `7f1a36036abc4e9b908a96b3107df5f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:40.437214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ff2e0a50-141a-468b-b41e-c47cdcb76e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:40.503873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 810/1619] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:40.505859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ec2d9-a92d-42d8-b42c-8f52423d5762 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.510523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 810/1620] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:40.511915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e0a50-141a-468b-b41e-c47cdcb76e2a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.572773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ec2d9-a92d-42d8-b42c-8f52423d5762 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.577743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb57ac2-f71c-40b0-bf09-68a6f3c914a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.578500 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb57ac2-f71c-40b0-bf09-68a6f3c914a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.578782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb57ac2-f71c-40b0-bf09-68a6f3c914a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.579058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb57ac2-f71c-40b0-bf09-68a6f3c914a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.587947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e0a50-141a-468b-b41e-c47cdcb76e2a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.594697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea9a33-ee5f-4cea-bb73-0d4c346a27ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.595779 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea9a33-ee5f-4cea-bb73-0d4c346a27ee admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.596396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea9a33-ee5f-4cea-bb73-0d4c346a27ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.596699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea9a33-ee5f-4cea-bb73-0d4c346a27ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.597350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea9a33-ee5f-4cea-bb73-0d4c346a27ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.599983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea9a33-ee5f-4cea-bb73-0d4c346a27ee admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.612728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 811/1621] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:40.612845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea9a33-ee5f-4cea-bb73-0d4c346a27ee admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.632089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:40.642450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb57ac2-f71c-40b0-bf09-68a6f3c914a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.643970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb57ac2-f71c-40b0-bf09-68a6f3c914a4 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.653791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 811/1622] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:40.654663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb57ac2-f71c-40b0-bf09-68a6f3c914a4 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.684559 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea9a33-ee5f-4cea-bb73-0d4c346a27ee admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.687711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55a2c9-92ed-4910-886a-c5ccc5cfecf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.688107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:40.688334 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55a2c9-92ed-4910-886a-c5ccc5cfecf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.689005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55a2c9-92ed-4910-886a-c5ccc5cfecf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.689273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55a2c9-92ed-4910-886a-c5ccc5cfecf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.705995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55a2c9-92ed-4910-886a-c5ccc5cfecf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.707524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55a2c9-92ed-4910-886a-c5ccc5cfecf1 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.730860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 812/1623] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:40.733409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691de5b-33da-40ba-aa54-2f14d1c7eb9a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.733695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691de5b-33da-40ba-aa54-2f14d1c7eb9a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.734020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691de5b-33da-40ba-aa54-2f14d1c7eb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.739197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb57ac2-f71c-40b0-bf09-68a6f3c914a4 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.747209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d51fa-7188-4820-bc18-b9204487fb78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.749511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d51fa-7188-4820-bc18-b9204487fb78 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.749773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d51fa-7188-4820-bc18-b9204487fb78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.750273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d51fa-7188-4820-bc18-b9204487fb78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.764304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d51fa-7188-4820-bc18-b9204487fb78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.766389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d51fa-7188-4820-bc18-b9204487fb78 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.771555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3691de5b-33da-40ba-aa54-2f14d1c7eb9a None None] MFA Rules not processed for user `64db87b93e5544f1a6b8167e2003e200`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:40.773373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3691de5b-33da-40ba-aa54-2f14d1c7eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:40.779542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 812/1624] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:40.780732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d51fa-7188-4820-bc18-b9204487fb78 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.797012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d51fa-7188-4820-bc18-b9204487fb78 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.804219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea7478-6f32-44be-85a7-713e9256ec57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.805718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea7478-6f32-44be-85a7-713e9256ec57 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.809432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea7478-6f32-44be-85a7-713e9256ec57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.809864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea7478-6f32-44be-85a7-713e9256ec57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.810582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea7478-6f32-44be-85a7-713e9256ec57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.813211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea7478-6f32-44be-85a7-713e9256ec57 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.855047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 813/1625] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:40.855893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691de5b-33da-40ba-aa54-2f14d1c7eb9a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.867634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691de5b-33da-40ba-aa54-2f14d1c7eb9a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.872078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c8986-e854-429a-8d54-1c309be6e997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.872250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c8986-e854-429a-8d54-1c309be6e997 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.872596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c8986-e854-429a-8d54-1c309be6e997 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.872793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c8986-e854-429a-8d54-1c309be6e997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.887161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c8986-e854-429a-8d54-1c309be6e997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.888020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c8986-e854-429a-8d54-1c309be6e997 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.902939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 814/1626] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:40.903782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c8986-e854-429a-8d54-1c309be6e997 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.917167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c8986-e854-429a-8d54-1c309be6e997 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:40.921420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb77b32-4f71-4d5d-8432-34f3e742f09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:40.921980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb77b32-4f71-4d5d-8432-34f3e742f09f admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.922275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb77b32-4f71-4d5d-8432-34f3e742f09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.922651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb77b32-4f71-4d5d-8432-34f3e742f09f admin admin] PATH_INFO: `/v3/projects/c5ec4c189693458ab07bde1354c2a439/users/52d974fe2d8f46c0acaac2490ab833cf/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.925158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb77b32-4f71-4d5d-8432-34f3e742f09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ec4c189693458ab07bde1354c2a439, user_id=52d974fe2d8f46c0acaac2490ab833cf, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:40.927976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb77b32-4f71-4d5d-8432-34f3e742f09f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:40.948355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 813/1627] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:40.954235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f1ba7-da56-422b-9335-c198ec0bc0a6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.954519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f1ba7-da56-422b-9335-c198ec0bc0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.957374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f1ba7-da56-422b-9335-c198ec0bc0a6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.967377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 815/1628] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:40 2022] PUT /identity/v3/projects/c5ec4c189693458ab07bde1354c2a439/users/52d974fe2d8f46c0acaac2490ab833cf/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:40.968431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb77b32-4f71-4d5d-8432-34f3e742f09f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.969862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 814/1629] 10.222.0.83 () {54 vars in 962 bytes} [Wed Aug 24 11:24:40 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:40.971039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa11ac-7ac2-42b4-b974-d8003607c0fa None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.971316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa11ac-7ac2-42b4-b974-d8003607c0fa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.973999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa11ac-7ac2-42b4-b974-d8003607c0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.020847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-89aa11ac-7ac2-42b4-b974-d8003607c0fa None None] MFA Rules not processed for user `52d974fe2d8f46c0acaac2490ab833cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:41.023000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-89aa11ac-7ac2-42b4-b974-d8003607c0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:41.064753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb77b32-4f71-4d5d-8432-34f3e742f09f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:41.096524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:41.134147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77771c7-2713-46e6-89d9-5eb0f6c324ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:41.134770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77771c7-2713-46e6-89d9-5eb0f6c324ac admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.135048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77771c7-2713-46e6-89d9-5eb0f6c324ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.135315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77771c7-2713-46e6-89d9-5eb0f6c324ac admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.135958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77771c7-2713-46e6-89d9-5eb0f6c324ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:41.138866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77771c7-2713-46e6-89d9-5eb0f6c324ac admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:41.151386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 815/1630] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:41.154168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c24bc-72a5-4932-97c6-0406fecef59e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.154885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c24bc-72a5-4932-97c6-0406fecef59e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.155890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c24bc-72a5-4932-97c6-0406fecef59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.177639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0f4c24bc-72a5-4932-97c6-0406fecef59e None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:41.178943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0f4c24bc-72a5-4932-97c6-0406fecef59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:41.301690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 816/1631] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/users => generated 299 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:41.305557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 816/1632] 10.222.0.83 () {58 vars in 1059 bytes} [Wed Aug 24 11:24:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:41.315091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77771c7-2713-46e6-89d9-5eb0f6c324ac admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:41.315225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c24bc-72a5-4932-97c6-0406fecef59e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:41.332520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c24bc-72a5-4932-97c6-0406fecef59e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:41.339025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77771c7-2713-46e6-89d9-5eb0f6c324ac admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:41.341747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0e1b2-4aab-40c5-a565-26148a8e8b98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:41.342432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0e1b2-4aab-40c5-a565-26148a8e8b98 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.342756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0e1b2-4aab-40c5-a565-26148a8e8b98 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.343036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0e1b2-4aab-40c5-a565-26148a8e8b98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.344032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e379-27f3-4e66-8d3d-ddeb8b81cf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:41.344685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e379-27f3-4e66-8d3d-ddeb8b81cf72 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.345028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e379-27f3-4e66-8d3d-ddeb8b81cf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.345277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e379-27f3-4e66-8d3d-ddeb8b81cf72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.345980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e379-27f3-4e66-8d3d-ddeb8b81cf72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:41.347661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e379-27f3-4e66-8d3d-ddeb8b81cf72 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:41.366162 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:41.371514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 817/1633] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:41.376923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e379-27f3-4e66-8d3d-ddeb8b81cf72 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:41.387903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e379-27f3-4e66-8d3d-ddeb8b81cf72 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:41.393583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67036b10-aaff-4427-aa34-5a7aa79450a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:41.394463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67036b10-aaff-4427-aa34-5a7aa79450a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.394929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67036b10-aaff-4427-aa34-5a7aa79450a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.395220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67036b10-aaff-4427-aa34-5a7aa79450a3 admin admin] PATH_INFO: `/v3/projects/aa9c2c5da9224168879d73b0877c35f1/users/2a4aba4954bc4d049baa3a8802a91677/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.414237 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67036b10-aaff-4427-aa34-5a7aa79450a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9c2c5da9224168879d73b0877c35f1, user_id=2a4aba4954bc4d049baa3a8802a91677, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:41.417531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67036b10-aaff-4427-aa34-5a7aa79450a3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:41.431578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0e1b2-4aab-40c5-a565-26148a8e8b98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:41.433635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0e1b2-4aab-40c5-a565-26148a8e8b98 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:41.462324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 817/1634] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:24:41.536639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 818/1635] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:41 2022] PUT /identity/v3/projects/aa9c2c5da9224168879d73b0877c35f1/users/2a4aba4954bc4d049baa3a8802a91677/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:41.541071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0e1b2-4aab-40c5-a565-26148a8e8b98 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:41.556665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0e1b2-4aab-40c5-a565-26148a8e8b98 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:41.687849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b44cd-4b47-469b-8a4a-e0ad7d6b5937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:41.688784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b44cd-4b47-469b-8a4a-e0ad7d6b5937 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.689234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b44cd-4b47-469b-8a4a-e0ad7d6b5937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.689508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b44cd-4b47-469b-8a4a-e0ad7d6b5937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.690200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b44cd-4b47-469b-8a4a-e0ad7d6b5937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:41.691931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b44cd-4b47-469b-8a4a-e0ad7d6b5937 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:41.702838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 818/1636] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:41.709636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67036b10-aaff-4427-aa34-5a7aa79450a3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:41.727735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff611d9-74c2-4e74-878b-a957d3f9de84 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.728639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff611d9-74c2-4e74-878b-a957d3f9de84 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.728639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff611d9-74c2-4e74-878b-a957d3f9de84 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.731287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 819/1637] 10.222.0.83 () {54 vars in 912 bytes} [Wed Aug 24 11:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:41.738514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e7d4b-595a-4fd8-9a07-be24c8483bbe None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.738927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e7d4b-595a-4fd8-9a07-be24c8483bbe None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.739308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e7d4b-595a-4fd8-9a07-be24c8483bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.755906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67036b10-aaff-4427-aa34-5a7aa79450a3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:41.764831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3ada1-dd71-497a-9613-d46ddf1a7db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:41.765496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3ada1-dd71-497a-9613-d46ddf1a7db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.765786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3ada1-dd71-497a-9613-d46ddf1a7db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.766592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3ada1-dd71-497a-9613-d46ddf1a7db9 admin admin] PATH_INFO: `/v3/projects/aa9c2c5da9224168879d73b0877c35f1/users/2a4aba4954bc4d049baa3a8802a91677/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.768609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3ada1-dd71-497a-9613-d46ddf1a7db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9c2c5da9224168879d73b0877c35f1, user_id=2a4aba4954bc4d049baa3a8802a91677, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:41.771281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-637e7d4b-595a-4fd8-9a07-be24c8483bbe None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:41.771498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3ada1-dd71-497a-9613-d46ddf1a7db9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:41.774260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-637e7d4b-595a-4fd8-9a07-be24c8483bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:41.809907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 819/1638] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:41 2022] PUT /identity/v3/projects/aa9c2c5da9224168879d73b0877c35f1/users/2a4aba4954bc4d049baa3a8802a91677/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:41.819775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3ada1-dd71-497a-9613-d46ddf1a7db9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:41.902420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 820/1639] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:24:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:24:41.903610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f5d0c-9b2d-4a99-9148-50d4937fe345 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.903966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f5d0c-9b2d-4a99-9148-50d4937fe345 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.904313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f5d0c-9b2d-4a99-9148-50d4937fe345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.906034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3ada1-dd71-497a-9613-d46ddf1a7db9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:41.912101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31488b-372c-4ffe-a78f-2b7527cc2c28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:41.912836 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31488b-372c-4ffe-a78f-2b7527cc2c28 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.913478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31488b-372c-4ffe-a78f-2b7527cc2c28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.913665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31488b-372c-4ffe-a78f-2b7527cc2c28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.939729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-846f5d0c-9b2d-4a99-9148-50d4937fe345 None None] MFA Rules not processed for user `2a4aba4954bc4d049baa3a8802a91677`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:41.941117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-846f5d0c-9b2d-4a99-9148-50d4937fe345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:42.022479 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31488b-372c-4ffe-a78f-2b7527cc2c28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:42.024185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31488b-372c-4ffe-a78f-2b7527cc2c28 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:42.043207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:42.048050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 820/1640] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:24:42.058884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 821/1641] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:42.098732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31488b-372c-4ffe-a78f-2b7527cc2c28 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:42.117756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31488b-372c-4ffe-a78f-2b7527cc2c28 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:42.123307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03a49-5f5e-4db9-b679-278e74c533e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:42.123762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03a49-5f5e-4db9-b679-278e74c533e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:42.124351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03a49-5f5e-4db9-b679-278e74c533e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:42.124556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03a49-5f5e-4db9-b679-278e74c533e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:42.154113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:42.282845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f5d0c-9b2d-4a99-9148-50d4937fe345 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:42.294770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f5d0c-9b2d-4a99-9148-50d4937fe345 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:42.297777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d03a49-5f5e-4db9-b679-278e74c533e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:42.301456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d03a49-5f5e-4db9-b679-278e74c533e8 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:42.306120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755ae2b-4e00-4df3-ac12-7dcd1ccc8fb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:42.307083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755ae2b-4e00-4df3-ac12-7dcd1ccc8fb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:42.307538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755ae2b-4e00-4df3-ac12-7dcd1ccc8fb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:42.307980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755ae2b-4e00-4df3-ac12-7dcd1ccc8fb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:42.331533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 821/1642] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:42.332951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755ae2b-4e00-4df3-ac12-7dcd1ccc8fb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:42.334838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755ae2b-4e00-4df3-ac12-7dcd1ccc8fb3 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:42.367353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 822/1643] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:24:42.907593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03a49-5f5e-4db9-b679-278e74c533e8 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:42.911582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755ae2b-4e00-4df3-ac12-7dcd1ccc8fb3 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:42.911582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755ae2b-4e00-4df3-ac12-7dcd1ccc8fb3 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:42.911582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba3a2d-4e2b-4325-9cf2-69a49eb7c01f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:42.911582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba3a2d-4e2b-4325-9cf2-69a49eb7c01f service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:42.914736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba3a2d-4e2b-4325-9cf2-69a49eb7c01f service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:42.914736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba3a2d-4e2b-4325-9cf2-69a49eb7c01f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:42.999548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03a49-5f5e-4db9-b679-278e74c533e8 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:42.999548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:42.999548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:42.999548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:43.000774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] PATH_INFO: `/v3/users/7f1a36036abc4e9b908a96b3107df5f9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:43.000774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f1a36036abc4e9b908a96b3107df5f9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:43.000774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:24:43.000774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Aug 24 11:24:43.000774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: Aug 24 11:24:43.000774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:43.001234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba3a2d-4e2b-4325-9cf2-69a49eb7c01f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:43.001234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba3a2d-4e2b-4325-9cf2-69a49eb7c01f service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:43.001234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 823/1644] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:43.353484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f1a36036abc4e9b908a96b3107df5f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:43.356367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f1a36036abc4e9b908a96b3107df5f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:43.357505 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f1a36036abc4e9b908a96b3107df5f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:43.385015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f1a36036abc4e9b908a96b3107df5f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:43.437633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f1a36036abc4e9b908a96b3107df5f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:43.539017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 822/1645] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:24:42 2022] DELETE /identity/v3/users/7f1a36036abc4e9b908a96b3107df5f9 => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:43.540335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef0c5c-2905-48b3-8df1-240f7d4ff872 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:43.541051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef0c5c-2905-48b3-8df1-240f7d4ff872 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:43.541051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef0c5c-2905-48b3-8df1-240f7d4ff872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:43.583522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a8ef0c5c-2905-48b3-8df1-240f7d4ff872 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:43.585520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a8ef0c5c-2905-48b3-8df1-240f7d4ff872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:43.760606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 824/1646] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:24:43.791734 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:43.793771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:43.878160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:43.960277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cdc89-def4-4e43-8717-2245c24f1999 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:43.971008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adf7bf-967d-487f-9ee5-f03dc3d06d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:43.971806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adf7bf-967d-487f-9ee5-f03dc3d06d09 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:43.972169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adf7bf-967d-487f-9ee5-f03dc3d06d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:43.972566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adf7bf-967d-487f-9ee5-f03dc3d06d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:43.995472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adf7bf-967d-487f-9ee5-f03dc3d06d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:43.997217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adf7bf-967d-487f-9ee5-f03dc3d06d09 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:44.012116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 823/1647] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:44.968035 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef0c5c-2905-48b3-8df1-240f7d4ff872 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:44.970496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a8ef0c5c-2905-48b3-8df1-240f7d4ff872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:45.003687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef0c5c-2905-48b3-8df1-240f7d4ff872 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:45.026046 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:45.026703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:45.026983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:45.027257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] PATH_INFO: `/v3/projects/b335b0177512496c9e6c1aa4b4f8532f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:45.029292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b335b0177512496c9e6c1aa4b4f8532f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:45.030398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:24:45.030398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Aug 24 11:24:45.030398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: Aug 24 11:24:45.035048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:45.299788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b335b0177512496c9e6c1aa4b4f8532f', 'request_id': 'req-367302d5-3f86-433b-a2bc-0ccb3b984c41'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:45.310738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:45.328615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b335b0177512496c9e6c1aa4b4f8532f', 'request_id': 'req-367302d5-3f86-433b-a2bc-0ccb3b984c41'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:45.399881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b335b0177512496c9e6c1aa4b4f8532f', 'request_id': 'req-367302d5-3f86-433b-a2bc-0ccb3b984c41'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:45.498185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] The token cache is being invalidate because project b335b0177512496c9e6c1aa4b4f8532f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:24:45.498655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:45.501809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:45.506183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 825/1648] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:24:44 2022] DELETE /identity/v3/projects/b335b0177512496c9e6c1aa4b4f8532f => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:45.510105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adf7bf-967d-487f-9ee5-f03dc3d06d09 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:45.526147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adf7bf-967d-487f-9ee5-f03dc3d06d09 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:45.664862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:45.665527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:45.666126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:45.667188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] PATH_INFO: `/v3/users/2a4aba4954bc4d049baa3a8802a91677` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:45.671888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4aba4954bc4d049baa3a8802a91677)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:45.680650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:45.921198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4aba4954bc4d049baa3a8802a91677'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:45.922113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4aba4954bc4d049baa3a8802a91677'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:45.923065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4aba4954bc4d049baa3a8802a91677'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:45.946945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4aba4954bc4d049baa3a8802a91677'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:45.989234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-26ad7b0e-554c-4525-90b5-1291a82d1031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4aba4954bc4d049baa3a8802a91677'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:46.100951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 824/1649] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:24:45 2022] DELETE /identity/v3/users/2a4aba4954bc4d049baa3a8802a91677 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:48.139193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:48.140199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:48.160978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367302d5-3f86-433b-a2bc-0ccb3b984c41 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:48.313486 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:48.314219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:48.314444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:48.314720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] PATH_INFO: `/v3/projects/aa9c2c5da9224168879d73b0877c35f1` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:48.319351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9c2c5da9224168879d73b0877c35f1)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:48.324024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:48.495266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9c2c5da9224168879d73b0877c35f1', 'request_id': 'req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:48.502548 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9c2c5da9224168879d73b0877c35f1', 'request_id': 'req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:48.519852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9c2c5da9224168879d73b0877c35f1', 'request_id': 'req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:48.594667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] The token cache is being invalidate because project aa9c2c5da9224168879d73b0877c35f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:24:48.596968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:48.596968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ea636b5-62f5-4981-a3fd-3b2bf68722f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:48.597455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 826/1650] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:24:48 2022] DELETE /identity/v3/projects/aa9c2c5da9224168879d73b0877c35f1 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:48.602139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bf23d-4e1a-413f-b295-74684ccaace9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:48.602699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bf23d-4e1a-413f-b295-74684ccaace9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:48.602699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bf23d-4e1a-413f-b295-74684ccaace9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:48.626338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2e0bf23d-4e1a-413f-b295-74684ccaace9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:48.627971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2e0bf23d-4e1a-413f-b295-74684ccaace9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:48.680021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:48.702926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 825/1651] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:48.710753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f335ee-bfd9-4475-8911-9e7c317d1b4b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:48.711279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f335ee-bfd9-4475-8911-9e7c317d1b4b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:48.711763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f335ee-bfd9-4475-8911-9e7c317d1b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:48.736517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c4f335ee-bfd9-4475-8911-9e7c317d1b4b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:48.738193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c4f335ee-bfd9-4475-8911-9e7c317d1b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:48.752274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 827/1652] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:48.757578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bf23d-4e1a-413f-b295-74684ccaace9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:48.769645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bf23d-4e1a-413f-b295-74684ccaace9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:48.781783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cdf49-cb00-4dd2-ac54-0f1f600ea839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:48.782813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cdf49-cb00-4dd2-ac54-0f1f600ea839 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:48.783361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cdf49-cb00-4dd2-ac54-0f1f600ea839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:48.783838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cdf49-cb00-4dd2-ac54-0f1f600ea839 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:48.784729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cdf49-cb00-4dd2-ac54-0f1f600ea839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:48.785227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cdf49-cb00-4dd2-ac54-0f1f600ea839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:48.788634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cdf49-cb00-4dd2-ac54-0f1f600ea839 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:48.824965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 826/1653] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:24:48.826234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f335ee-bfd9-4475-8911-9e7c317d1b4b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:48.840493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f335ee-bfd9-4475-8911-9e7c317d1b4b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:48.854572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b32ff-f72c-4e1b-8d72-47b0dc0828f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:48.855199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b32ff-f72c-4e1b-8d72-47b0dc0828f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:48.855481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b32ff-f72c-4e1b-8d72-47b0dc0828f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:48.855754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b32ff-f72c-4e1b-8d72-47b0dc0828f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:48.856390 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b32ff-f72c-4e1b-8d72-47b0dc0828f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:48.858878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b32ff-f72c-4e1b-8d72-47b0dc0828f1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:48.981059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 828/1654] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:48 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:48.985405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cdf49-cb00-4dd2-ac54-0f1f600ea839 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:48.998475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cdf49-cb00-4dd2-ac54-0f1f600ea839 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:49.133992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97fc25-7f63-4a77-be20-60eda2f4a792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:49.135793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97fc25-7f63-4a77-be20-60eda2f4a792 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:49.136340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97fc25-7f63-4a77-be20-60eda2f4a792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:49.136816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97fc25-7f63-4a77-be20-60eda2f4a792 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:49.137678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97fc25-7f63-4a77-be20-60eda2f4a792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:49.144480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97fc25-7f63-4a77-be20-60eda2f4a792 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:49.299764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b32ff-f72c-4e1b-8d72-47b0dc0828f1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:49.302800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 827/1655] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:48 2022] POST /identity/v3/users => generated 290 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:49.336168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b32ff-f72c-4e1b-8d72-47b0dc0828f1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:49.358679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449e41c-1bfc-4058-a67b-be6fa69deb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:49.359702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449e41c-1bfc-4058-a67b-be6fa69deb92 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:49.362980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449e41c-1bfc-4058-a67b-be6fa69deb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:49.363462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449e41c-1bfc-4058-a67b-be6fa69deb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:49.364679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449e41c-1bfc-4058-a67b-be6fa69deb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:49.367343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449e41c-1bfc-4058-a67b-be6fa69deb92 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:49.394442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 829/1656] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:49.403475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97fc25-7f63-4a77-be20-60eda2f4a792 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:49.442112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97fc25-7f63-4a77-be20-60eda2f4a792 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:49.454670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a76ce2-dc5b-4e2b-9bb4-a5cf9e0e26cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:49.456302 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76ce2-dc5b-4e2b-9bb4-a5cf9e0e26cc admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:49.457203 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76ce2-dc5b-4e2b-9bb4-a5cf9e0e26cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:49.457487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76ce2-dc5b-4e2b-9bb4-a5cf9e0e26cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:49.459434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a76ce2-dc5b-4e2b-9bb4-a5cf9e0e26cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:49.462687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a76ce2-dc5b-4e2b-9bb4-a5cf9e0e26cc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:49.504138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449e41c-1bfc-4058-a67b-be6fa69deb92 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:49.506103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 828/1657] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:49.523163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449e41c-1bfc-4058-a67b-be6fa69deb92 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:49.538296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96242fee-5d3c-4d08-8cb0-c1e7da693d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:49.540124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242fee-5d3c-4d08-8cb0-c1e7da693d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:49.540411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242fee-5d3c-4d08-8cb0-c1e7da693d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:49.540956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242fee-5d3c-4d08-8cb0-c1e7da693d82 admin admin] PATH_INFO: `/v3/projects/39f957195bd441b8ae66d8febd3f9677/users/ae227a304d9f493bbd24fa895a5369e6/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:49.584247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96242fee-5d3c-4d08-8cb0-c1e7da693d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f957195bd441b8ae66d8febd3f9677, user_id=ae227a304d9f493bbd24fa895a5369e6, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:49.587701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96242fee-5d3c-4d08-8cb0-c1e7da693d82 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:49.730747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8b359-1950-4885-9b2a-ee023d1b5efa None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:49.732144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8b359-1950-4885-9b2a-ee023d1b5efa None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:49.732937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8b359-1950-4885-9b2a-ee023d1b5efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:49.737050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 830/1658] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:49 2022] PUT /identity/v3/projects/39f957195bd441b8ae66d8febd3f9677/users/ae227a304d9f493bbd24fa895a5369e6/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:24:49.788245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4af8b359-1950-4885-9b2a-ee023d1b5efa None None] MFA Rules not processed for user `ae227a304d9f493bbd24fa895a5369e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:49.791564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4af8b359-1950-4885-9b2a-ee023d1b5efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:50.053320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 829/1659] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:50.057657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96242fee-5d3c-4d08-8cb0-c1e7da693d82 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:50.082040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:50.211270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96242fee-5d3c-4d08-8cb0-c1e7da693d82 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:50.228387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced19ef-214d-4628-9881-d6e41cb5aa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:50.229057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced19ef-214d-4628-9881-d6e41cb5aa36 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:50.229337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced19ef-214d-4628-9881-d6e41cb5aa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:50.229607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced19ef-214d-4628-9881-d6e41cb5aa36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:50.230283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced19ef-214d-4628-9881-d6e41cb5aa36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:50.233643 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced19ef-214d-4628-9881-d6e41cb5aa36 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:50.376206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8b359-1950-4885-9b2a-ee023d1b5efa None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:50.378617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 831/1660] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:50 2022] POST /identity/v3/projects => generated 379 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:50.394322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8b359-1950-4885-9b2a-ee023d1b5efa None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:50.484265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19253fd3-bdc6-4b56-86a4-d9d8972d3594 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:50.485181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19253fd3-bdc6-4b56-86a4-d9d8972d3594 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:50.485988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19253fd3-bdc6-4b56-86a4-d9d8972d3594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:50.505966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:50.544073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e5653-fb5b-4a61-b157-7942ff91e79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:50.544690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e5653-fb5b-4a61-b157-7942ff91e79c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:50.544978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e5653-fb5b-4a61-b157-7942ff91e79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:50.545251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e5653-fb5b-4a61-b157-7942ff91e79c admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:50.545956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e5653-fb5b-4a61-b157-7942ff91e79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:50.547977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e5653-fb5b-4a61-b157-7942ff91e79c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:50.605385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-19253fd3-bdc6-4b56-86a4-d9d8972d3594 None None] MFA Rules not processed for user `cd79bc9c3a694fe2a4e3c645e682e438`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:50.612436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-19253fd3-bdc6-4b56-86a4-d9d8972d3594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:50.652294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 830/1661] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:50 2022] POST /identity/v3/users => generated 296 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:24:50.656492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e5653-fb5b-4a61-b157-7942ff91e79c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:50.674493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e5653-fb5b-4a61-b157-7942ff91e79c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:50.695367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b87e7-3037-4689-93e3-978982311ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:50.696267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b87e7-3037-4689-93e3-978982311ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:50.696565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b87e7-3037-4689-93e3-978982311ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:50.696840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b87e7-3037-4689-93e3-978982311ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:50.701649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b87e7-3037-4689-93e3-978982311ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:50.703433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b87e7-3037-4689-93e3-978982311ac3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:50.738746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 831/1662] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:50.743266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b87e7-3037-4689-93e3-978982311ac3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:50.769498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b87e7-3037-4689-93e3-978982311ac3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:50.781647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 832/1663] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:24:50.794759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b4210-5147-461a-b15a-d28c47179894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:50.795360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b4210-5147-461a-b15a-d28c47179894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:50.795682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b4210-5147-461a-b15a-d28c47179894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:50.796048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b4210-5147-461a-b15a-d28c47179894 admin admin] PATH_INFO: `/v3/projects/ce61c77e10264fd09ff58d128ea029c3/users/f256cfe0123942cb815268c00642feee/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:50.797434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19253fd3-bdc6-4b56-86a4-d9d8972d3594 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:50.799076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-19253fd3-bdc6-4b56-86a4-d9d8972d3594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:50.831799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b4210-5147-461a-b15a-d28c47179894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce61c77e10264fd09ff58d128ea029c3, user_id=f256cfe0123942cb815268c00642feee, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:50.835493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b4210-5147-461a-b15a-d28c47179894 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:50.945765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 832/1664] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:50 2022] PUT /identity/v3/projects/ce61c77e10264fd09ff58d128ea029c3/users/f256cfe0123942cb815268c00642feee/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:50.949698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19253fd3-bdc6-4b56-86a4-d9d8972d3594 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:50.950391 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b4210-5147-461a-b15a-d28c47179894 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:51.081964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa9556-f021-441e-8fe3-bb6b1a0fd440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:51.083260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa9556-f021-441e-8fe3-bb6b1a0fd440 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:51.083538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa9556-f021-441e-8fe3-bb6b1a0fd440 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:51.083858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa9556-f021-441e-8fe3-bb6b1a0fd440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:51.106037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b4210-5147-461a-b15a-d28c47179894 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:51.134470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a0b5b-684d-4ea8-8171-200684dcba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:51.135137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a0b5b-684d-4ea8-8171-200684dcba5f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:51.135453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a0b5b-684d-4ea8-8171-200684dcba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:51.135710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a0b5b-684d-4ea8-8171-200684dcba5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:51.136387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a0b5b-684d-4ea8-8171-200684dcba5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:51.138101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a0b5b-684d-4ea8-8171-200684dcba5f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:51.150190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 833/1665] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:24:51.154709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a0b5b-684d-4ea8-8171-200684dcba5f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:51.172581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a0b5b-684d-4ea8-8171-200684dcba5f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:51.184857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443cdbc-71dd-4c0b-986f-e7fe36b6fdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:51.187280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443cdbc-71dd-4c0b-986f-e7fe36b6fdc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:51.187670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443cdbc-71dd-4c0b-986f-e7fe36b6fdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:51.188092 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443cdbc-71dd-4c0b-986f-e7fe36b6fdc1 admin admin] PATH_INFO: `/v3/projects/ce61c77e10264fd09ff58d128ea029c3/users/f256cfe0123942cb815268c00642feee/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:51.190714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443cdbc-71dd-4c0b-986f-e7fe36b6fdc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce61c77e10264fd09ff58d128ea029c3, user_id=f256cfe0123942cb815268c00642feee, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:51.195109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443cdbc-71dd-4c0b-986f-e7fe36b6fdc1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:51.227671 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 834/1666] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:24:51 2022] PUT /identity/v3/projects/ce61c77e10264fd09ff58d128ea029c3/users/f256cfe0123942cb815268c00642feee/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:24:51.229233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd01294-721e-4f50-bbc6-ff7f66d4c92a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:51.229416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd01294-721e-4f50-bbc6-ff7f66d4c92a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:51.229647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd01294-721e-4f50-bbc6-ff7f66d4c92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:51.238878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa9556-f021-441e-8fe3-bb6b1a0fd440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:51.240543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa9556-f021-441e-8fe3-bb6b1a0fd440 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:51.264815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 833/1667] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:51.294432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2fd01294-721e-4f50-bbc6-ff7f66d4c92a None None] MFA Rules not processed for user `f256cfe0123942cb815268c00642feee`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:51.298189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2fd01294-721e-4f50-bbc6-ff7f66d4c92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:51.435706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 835/1668] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:51.451175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa9556-f021-441e-8fe3-bb6b1a0fd440 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:51.453679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-eeaa9556-f021-441e-8fe3-bb6b1a0fd440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:51.608235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa9556-f021-441e-8fe3-bb6b1a0fd440 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:51.697069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3010b-1b60-4651-a9a1-06a9925498a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:51.703237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3010b-1b60-4651-a9a1-06a9925498a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:51.703237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3010b-1b60-4651-a9a1-06a9925498a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:51.703237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3010b-1b60-4651-a9a1-06a9925498a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:51.726425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:51.740357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3010b-1b60-4651-a9a1-06a9925498a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:51.745853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3010b-1b60-4651-a9a1-06a9925498a2 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:51.781321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 834/1669] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:24:52.242934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac5592-7e81-410d-a258-2a1fa910b35c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:52.247870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac5592-7e81-410d-a258-2a1fa910b35c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:52.250078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac5592-7e81-410d-a258-2a1fa910b35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:52.260052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:52.303292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-edac5592-7e81-410d-a258-2a1fa910b35c None None] MFA Rules not processed for user `4aa39f218d4d463291ffe15b929c4e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:52.304796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-edac5592-7e81-410d-a258-2a1fa910b35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:52.415372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 836/1670] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:52.426833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e46c07-889b-4524-965f-2882a2b13365 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:52.427178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e46c07-889b-4524-965f-2882a2b13365 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:52.427446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e46c07-889b-4524-965f-2882a2b13365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:52.462795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-66e46c07-889b-4524-965f-2882a2b13365 None None] MFA Rules not processed for user `4aa39f218d4d463291ffe15b929c4e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:52.464389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-66e46c07-889b-4524-965f-2882a2b13365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:52.502715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 835/1671] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:52.516569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac5592-7e81-410d-a258-2a1fa910b35c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:52.714140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac5592-7e81-410d-a258-2a1fa910b35c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:52.721030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fafc7-7a9e-41cd-acdb-5d0be45056c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:52.721743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fafc7-7a9e-41cd-acdb-5d0be45056c2 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:52.722099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fafc7-7a9e-41cd-acdb-5d0be45056c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:52.722420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fafc7-7a9e-41cd-acdb-5d0be45056c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:52.735238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fafc7-7a9e-41cd-acdb-5d0be45056c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:52.737005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fafc7-7a9e-41cd-acdb-5d0be45056c2 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:52.779167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 837/1672] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:52.888247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945ff37-876c-4415-a997-c1512ac17255 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:52.888247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945ff37-876c-4415-a997-c1512ac17255 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:52.888528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945ff37-876c-4415-a997-c1512ac17255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:52.922271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:52.936629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6945ff37-876c-4415-a997-c1512ac17255 None None] MFA Rules not processed for user `92712c36310c440c94da6c432c77a895`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:52.938142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6945ff37-876c-4415-a997-c1512ac17255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:53.109887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 836/1673] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:53.119134 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fafc7-7a9e-41cd-acdb-5d0be45056c2 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:53.121319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b10fafc7-7a9e-41cd-acdb-5d0be45056c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:53.165231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fafc7-7a9e-41cd-acdb-5d0be45056c2 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:53.178751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6351d-af45-4cea-8811-b2b053931caf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:53.184314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6351d-af45-4cea-8811-b2b053931caf service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:53.190004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6351d-af45-4cea-8811-b2b053931caf service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:53.190356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6351d-af45-4cea-8811-b2b053931caf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:53.202925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6351d-af45-4cea-8811-b2b053931caf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:53.210101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6351d-af45-4cea-8811-b2b053931caf service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:53.229151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 838/1674] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:53.312354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945ff37-876c-4415-a997-c1512ac17255 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:53.347303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945ff37-876c-4415-a997-c1512ac17255 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:53.353273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a25b4-e846-44a9-9377-ad6e00288fac tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f256cfe0123942cb815268c00642feee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce61c77e10264fd09ff58d128ea029c3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:53.354326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a25b4-e846-44a9-9377-ad6e00288fac tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:53.354831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a25b4-e846-44a9-9377-ad6e00288fac tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:53.355319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a25b4-e846-44a9-9377-ad6e00288fac tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:53.356230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a25b4-e846-44a9-9377-ad6e00288fac tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:53.356769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a25b4-e846-44a9-9377-ad6e00288fac tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:53.359239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a25b4-e846-44a9-9377-ad6e00288fac tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:53.407478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6351d-af45-4cea-8811-b2b053931caf service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:53.407943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 837/1675] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:53.422685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6351d-af45-4cea-8811-b2b053931caf service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:53.440335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451108d4-1dbf-4ad7-bfae-d848724f18fb tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f256cfe0123942cb815268c00642feee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce61c77e10264fd09ff58d128ea029c3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:53.440997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451108d4-1dbf-4ad7-bfae-d848724f18fb tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:53.441295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451108d4-1dbf-4ad7-bfae-d848724f18fb tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:53.441561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451108d4-1dbf-4ad7-bfae-d848724f18fb tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:53.449703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451108d4-1dbf-4ad7-bfae-d848724f18fb tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:53.465167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451108d4-1dbf-4ad7-bfae-d848724f18fb tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:53.651993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 839/1676] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:53 2022] POST /identity/v3/projects => generated 352 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:24:53.657012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a25b4-e846-44a9-9377-ad6e00288fac tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:53.669491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a25b4-e846-44a9-9377-ad6e00288fac tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:53.809071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215e314-d3f1-4aec-9732-8c7cbfc5ed04 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:53.809813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215e314-d3f1-4aec-9732-8c7cbfc5ed04 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:53.810397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215e314-d3f1-4aec-9732-8c7cbfc5ed04 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:53.813111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 840/1677] 10.222.0.83 () {54 vars in 912 bytes} [Wed Aug 24 11:24:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:53.818879 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8780ef-7d1d-400a-b8a5-9e789708ad93 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:53.819403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8780ef-7d1d-400a-b8a5-9e789708ad93 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:53.819827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8780ef-7d1d-400a-b8a5-9e789708ad93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:53.827249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5652782-9cf3-43f7-a46a-911c0795d7f5 tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f256cfe0123942cb815268c00642feee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce61c77e10264fd09ff58d128ea029c3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:53.828375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5652782-9cf3-43f7-a46a-911c0795d7f5 tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:53.828782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5652782-9cf3-43f7-a46a-911c0795d7f5 tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:53.829020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5652782-9cf3-43f7-a46a-911c0795d7f5 tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:53.829695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5652782-9cf3-43f7-a46a-911c0795d7f5 tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:53.830041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5652782-9cf3-43f7-a46a-911c0795d7f5 tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:53.837478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5652782-9cf3-43f7-a46a-911c0795d7f5 tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:53.870594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 838/1678] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:53.923170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ed8780ef-7d1d-400a-b8a5-9e789708ad93 None None] MFA Rules not processed for user `b0145242ef784168bada8348ccd95f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:53.923170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ed8780ef-7d1d-400a-b8a5-9e789708ad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:53.946647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5652782-9cf3-43f7-a46a-911c0795d7f5 tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:53.956094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5652782-9cf3-43f7-a46a-911c0795d7f5 tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:53.964997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f256cfe0123942cb815268c00642feee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce61c77e10264fd09ff58d128ea029c3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:53.966283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:53.966810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:53.968021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] PATH_INFO: `/v3/projects/ef6e7cf0819740f5bdc330b20df96ce7` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:53.969427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ef6e7cf0819740f5bdc330b20df96ce7)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:53.972091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:53.979274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:54.046258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 841/1679] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:54.101307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8780ef-7d1d-400a-b8a5-9e789708ad93 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:54.104193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ed8780ef-7d1d-400a-b8a5-9e789708ad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:54.120740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8780ef-7d1d-400a-b8a5-9e789708ad93 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:54.133413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522212f-4155-4e3d-bbea-9829a3a38d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:54.134489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522212f-4155-4e3d-bbea-9829a3a38d10 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:54.138156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522212f-4155-4e3d-bbea-9829a3a38d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:54.138754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522212f-4155-4e3d-bbea-9829a3a38d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:54.165353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522212f-4155-4e3d-bbea-9829a3a38d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:54.165938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6e7cf0819740f5bdc330b20df96ce7', 'request_id': 'req-46425886-d050-45fe-9730-2233c6588dbd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:54.167399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522212f-4155-4e3d-bbea-9829a3a38d10 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:54.179694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6e7cf0819740f5bdc330b20df96ce7', 'request_id': 'req-46425886-d050-45fe-9730-2233c6588dbd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:54.188844 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 842/1680] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:54.198709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6e7cf0819740f5bdc330b20df96ce7', 'request_id': 'req-46425886-d050-45fe-9730-2233c6588dbd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:54.300147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] The token cache is being invalidate because project ef6e7cf0819740f5bdc330b20df96ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:24:54.300147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:54.300147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-46425886-d050-45fe-9730-2233c6588dbd tempest-VolumeQuotaClassesTest-103653882 tempest-VolumeQuotaClassesTest-103653882-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:54.306162 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 839/1681] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:24:53 2022] DELETE /identity/v3/projects/ef6e7cf0819740f5bdc330b20df96ce7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:24:55.242598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b80266-ec52-45af-ba9a-5e29b97da5ad None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:55.243311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b80266-ec52-45af-ba9a-5e29b97da5ad None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:55.243311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b80266-ec52-45af-ba9a-5e29b97da5ad None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:55.244795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 843/1682] 10.222.0.83 () {54 vars in 961 bytes} [Wed Aug 24 11:24:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:55.253098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c3165-6a56-476b-9aee-0b80d8bacc13 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:55.253807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c3165-6a56-476b-9aee-0b80d8bacc13 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:55.254125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c3165-6a56-476b-9aee-0b80d8bacc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:55.265674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:55.309496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2c5c3165-6a56-476b-9aee-0b80d8bacc13 None None] MFA Rules not processed for user `eb76ad667a744126ab00b38c346e9017`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:55.310791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2c5c3165-6a56-476b-9aee-0b80d8bacc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:55.328159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b80266-ec52-45af-ba9a-5e29b97da5ad None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:55.330735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-93b80266-ec52-45af-ba9a-5e29b97da5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:55.437705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 840/1683] 10.222.0.83 () {58 vars in 1058 bytes} [Wed Aug 24 11:24:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:55.445345 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c3165-6a56-476b-9aee-0b80d8bacc13 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:55.448499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b80266-ec52-45af-ba9a-5e29b97da5ad None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:55.453212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:55.454038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:55.454397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:55.454720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] PATH_INFO: `/v3/users/ae227a304d9f493bbd24fa895a5369e6` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:55.457232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae227a304d9f493bbd24fa895a5369e6)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:55.459933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:55.461930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c3165-6a56-476b-9aee-0b80d8bacc13 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:55.471406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1da7a-ad6e-4752-b3cb-e5c472a2a5a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:55.472967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1da7a-ad6e-4752-b3cb-e5c472a2a5a7 service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:55.473438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1da7a-ad6e-4752-b3cb-e5c472a2a5a7 service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:55.474338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1da7a-ad6e-4752-b3cb-e5c472a2a5a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:55.477427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7db1da7a-ad6e-4752-b3cb-e5c472a2a5a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:55.501233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1da7a-ad6e-4752-b3cb-e5c472a2a5a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:55.503354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1da7a-ad6e-4752-b3cb-e5c472a2a5a7 service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:55.519157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 841/1684] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:55.642457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae227a304d9f493bbd24fa895a5369e6'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:55.643135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae227a304d9f493bbd24fa895a5369e6'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:55.643750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae227a304d9f493bbd24fa895a5369e6'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:55.655785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae227a304d9f493bbd24fa895a5369e6'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:55.670570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae227a304d9f493bbd24fa895a5369e6'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:55.810833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 844/1685] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:24:55 2022] DELETE /identity/v3/users/ae227a304d9f493bbd24fa895a5369e6 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:55.841209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbab6d9-abe9-4650-b508-4005c299fb85 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:55.850879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbab6d9-abe9-4650-b508-4005c299fb85 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:55.850879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbab6d9-abe9-4650-b508-4005c299fb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:55.865382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6dbab6d9-abe9-4650-b508-4005c299fb85 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:55.866822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6dbab6d9-abe9-4650-b508-4005c299fb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:55.951917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 842/1686] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:55.959967 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:55.962097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:56.050705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:56.059509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62106b-96b5-4c71-ada2-03e3e3cdaa8e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:56.064009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bbacc-a63f-4c82-b573-e38669572b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:56.064985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bbacc-a63f-4c82-b573-e38669572b1c service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:56.065387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bbacc-a63f-4c82-b573-e38669572b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:56.065719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bbacc-a63f-4c82-b573-e38669572b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:56.106655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bbacc-a63f-4c82-b573-e38669572b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:56.109021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bbacc-a63f-4c82-b573-e38669572b1c service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:56.125954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 845/1687] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:56.295317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fa656-affb-46bf-ae22-461f29a48545 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:56.296967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fa656-affb-46bf-ae22-461f29a48545 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:56.297378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fa656-affb-46bf-ae22-461f29a48545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:56.332443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e22fa656-affb-46bf-ae22-461f29a48545 None None] MFA Rules not processed for user `cd79bc9c3a694fe2a4e3c645e682e438`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:56.334903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e22fa656-affb-46bf-ae22-461f29a48545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:56.497831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 843/1688] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:56.949228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bbacc-a63f-4c82-b573-e38669572b1c service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:56.953429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e56bbacc-a63f-4c82-b573-e38669572b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:56.972380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bbacc-a63f-4c82-b573-e38669572b1c service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:56.985558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:56.986460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:56.986890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:56.987327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] PATH_INFO: `/v3/projects/39f957195bd441b8ae66d8febd3f9677` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:56.989799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39f957195bd441b8ae66d8febd3f9677)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:56.992181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:57.162846 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39f957195bd441b8ae66d8febd3f9677', 'request_id': 'req-dc499307-47ea-4a76-979d-3114ad2f2e13'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.182545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39f957195bd441b8ae66d8febd3f9677', 'request_id': 'req-dc499307-47ea-4a76-979d-3114ad2f2e13'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.268132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09662-3923-49e2-9ff8-2b1630701480 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:57.268533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09662-3923-49e2-9ff8-2b1630701480 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:57.268948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09662-3923-49e2-9ff8-2b1630701480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:57.285544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39f957195bd441b8ae66d8febd3f9677', 'request_id': 'req-dc499307-47ea-4a76-979d-3114ad2f2e13'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.305901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] The token cache is being invalidate because project 39f957195bd441b8ae66d8febd3f9677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:24:57.305901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.306783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.307806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e0f09662-3923-49e2-9ff8-2b1630701480 None None] MFA Rules not processed for user `92712c36310c440c94da6c432c77a895`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:57.309315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 846/1689] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:24:56 2022] DELETE /identity/v3/projects/39f957195bd441b8ae66d8febd3f9677 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:57.313777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:57.315625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:57.317240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e0f09662-3923-49e2-9ff8-2b1630701480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:57.375629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:57.464734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 844/1690] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:57.501588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc499307-47ea-4a76-979d-3114ad2f2e13 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:57.507179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:57.507839 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:57.508121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:57.508397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] PATH_INFO: `/v3/users/f256cfe0123942cb815268c00642feee` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:57.509628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f256cfe0123942cb815268c00642feee)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:57.511844 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:57.760397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f256cfe0123942cb815268c00642feee'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.761438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f256cfe0123942cb815268c00642feee'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.762043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f256cfe0123942cb815268c00642feee'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.787106 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f256cfe0123942cb815268c00642feee'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.809327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a668153-544a-4341-b2f2-ff592b743b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f256cfe0123942cb815268c00642feee'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:57.811850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:57.902730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 847/1691] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:24:57 2022] DELETE /identity/v3/users/f256cfe0123942cb815268c00642feee => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:58.456157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d95a0-065f-4123-b367-255333308d96 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:58.456157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d95a0-065f-4123-b367-255333308d96 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:58.456157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d95a0-065f-4123-b367-255333308d96 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:58.459122 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 845/1692] 10.222.0.83 () {54 vars in 912 bytes} [Wed Aug 24 11:24:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:58.466977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d909a-3dc9-4f3b-8169-3ba97b7238b3 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:58.467925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d909a-3dc9-4f3b-8169-3ba97b7238b3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:58.468362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d909a-3dc9-4f3b-8169-3ba97b7238b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:58.505286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-822d909a-3dc9-4f3b-8169-3ba97b7238b3 None None] MFA Rules not processed for user `b0145242ef784168bada8348ccd95f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:58.506647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-822d909a-3dc9-4f3b-8169-3ba97b7238b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:58.609267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 848/1693] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:24:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:58.633049 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d95a0-065f-4123-b367-255333308d96 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:58.634856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8e2d95a0-065f-4123-b367-255333308d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:58.658260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d95a0-065f-4123-b367-255333308d96 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:58.669941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b49d6-6bc5-4e80-a432-f2fe582df0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:58.671325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b49d6-6bc5-4e80-a432-f2fe582df0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:58.672021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b49d6-6bc5-4e80-a432-f2fe582df0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:58.672449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b49d6-6bc5-4e80-a432-f2fe582df0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:58.706999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b49d6-6bc5-4e80-a432-f2fe582df0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:58.710844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b49d6-6bc5-4e80-a432-f2fe582df0f8 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:58.730493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 846/1694] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:58.822333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d909a-3dc9-4f3b-8169-3ba97b7238b3 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:58.827351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-822d909a-3dc9-4f3b-8169-3ba97b7238b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:58.931734 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d909a-3dc9-4f3b-8169-3ba97b7238b3 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:58.950912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:58.951608 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:58.953029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:58.953317 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] PATH_INFO: `/v3/projects/ce61c77e10264fd09ff58d128ea029c3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:58.954679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce61c77e10264fd09ff58d128ea029c3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:58.956967 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:59.110541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce61c77e10264fd09ff58d128ea029c3', 'request_id': 'req-421f4a3e-33f9-4161-a0d1-e268efe2326b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:59.122047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce61c77e10264fd09ff58d128ea029c3', 'request_id': 'req-421f4a3e-33f9-4161-a0d1-e268efe2326b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:59.222731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce61c77e10264fd09ff58d128ea029c3', 'request_id': 'req-421f4a3e-33f9-4161-a0d1-e268efe2326b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:59.245971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] The token cache is being invalidate because project ce61c77e10264fd09ff58d128ea029c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:24:59.246413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:59.247539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-421f4a3e-33f9-4161-a0d1-e268efe2326b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:24:59.255107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cc64c-19d9-48c0-977e-6f62fdce9cba None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:59.255443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cc64c-19d9-48c0-977e-6f62fdce9cba None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:59.255725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cc64c-19d9-48c0-977e-6f62fdce9cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:59.266069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 849/1695] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:24:58 2022] DELETE /identity/v3/projects/ce61c77e10264fd09ff58d128ea029c3 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:24:59.298578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-679cc64c-19d9-48c0-977e-6f62fdce9cba None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:59.299870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-679cc64c-19d9-48c0-977e-6f62fdce9cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:59.493750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 847/1696] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:59.503339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a6ce3-211b-46bc-86fd-9f0c3a86b0f6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:59.504263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a6ce3-211b-46bc-86fd-9f0c3a86b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:59.505003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a6ce3-211b-46bc-86fd-9f0c3a86b0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:59.533300 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f36a6ce3-211b-46bc-86fd-9f0c3a86b0f6 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:59.534897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f36a6ce3-211b-46bc-86fd-9f0c3a86b0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:59.567748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 850/1697] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:59.570219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cc64c-19d9-48c0-977e-6f62fdce9cba None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:59.599482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cc64c-19d9-48c0-977e-6f62fdce9cba None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:59.634513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cccc9-063d-40bf-84ab-40a389af25be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:59.635822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cccc9-063d-40bf-84ab-40a389af25be admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:59.636377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cccc9-063d-40bf-84ab-40a389af25be admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:59.637128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cccc9-063d-40bf-84ab-40a389af25be admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:59.638470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cccc9-063d-40bf-84ab-40a389af25be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:24:59.638927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cccc9-063d-40bf-84ab-40a389af25be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:59.640748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cccc9-063d-40bf-84ab-40a389af25be admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:59.661715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 848/1698] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:24:59.662568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a6ce3-211b-46bc-86fd-9f0c3a86b0f6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:59.673666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a6ce3-211b-46bc-86fd-9f0c3a86b0f6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:59.685496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829709f-9f76-4127-ba7c-a3302daa1803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:59.686954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829709f-9f76-4127-ba7c-a3302daa1803 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:59.687243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829709f-9f76-4127-ba7c-a3302daa1803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:59.687520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829709f-9f76-4127-ba7c-a3302daa1803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:59.688178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829709f-9f76-4127-ba7c-a3302daa1803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:59.691706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829709f-9f76-4127-ba7c-a3302daa1803 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:59.874774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 851/1699] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:24:59 2022] POST /identity/v3/projects => generated 407 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:24:59.881314 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cccc9-063d-40bf-84ab-40a389af25be admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:59.895656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cccc9-063d-40bf-84ab-40a389af25be admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:59.970762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f908b-5e77-4d2f-bbec-b46bc88d8ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:59.971376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f908b-5e77-4d2f-bbec-b46bc88d8ebc admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:59.971658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f908b-5e77-4d2f-bbec-b46bc88d8ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:59.971935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f908b-5e77-4d2f-bbec-b46bc88d8ebc admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:59.972751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f908b-5e77-4d2f-bbec-b46bc88d8ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:59.974784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f908b-5e77-4d2f-bbec-b46bc88d8ebc admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:00.084526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 849/1700] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:24:59 2022] POST /identity/v3/users => generated 304 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:00.088843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829709f-9f76-4127-ba7c-a3302daa1803 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:00.095384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:00.102654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829709f-9f76-4127-ba7c-a3302daa1803 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:00.109837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c476ab-5995-4c18-9aab-e7e677459b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:00.110612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c476ab-5995-4c18-9aab-e7e677459b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:00.111209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c476ab-5995-4c18-9aab-e7e677459b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:00.111640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c476ab-5995-4c18-9aab-e7e677459b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:00.112745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c476ab-5995-4c18-9aab-e7e677459b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:00.114426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c476ab-5995-4c18-9aab-e7e677459b39 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:00.137093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 852/1701] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:00.141370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f908b-5e77-4d2f-bbec-b46bc88d8ebc admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:00.152962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f908b-5e77-4d2f-bbec-b46bc88d8ebc admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:00.158425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff97add-b7bd-4128-83fa-a802c3c80aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:00.165992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff97add-b7bd-4128-83fa-a802c3c80aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:00.165992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff97add-b7bd-4128-83fa-a802c3c80aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:00.170208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff97add-b7bd-4128-83fa-a802c3c80aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:00.171582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff97add-b7bd-4128-83fa-a802c3c80aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:00.173233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff97add-b7bd-4128-83fa-a802c3c80aaa admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:00.203121 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 850/1702] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:00.207631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c476ab-5995-4c18-9aab-e7e677459b39 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:00.226445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c476ab-5995-4c18-9aab-e7e677459b39 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:00.234796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92d496-011e-4814-a6b2-b552ed8616eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:00.240670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92d496-011e-4814-a6b2-b552ed8616eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:00.240670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92d496-011e-4814-a6b2-b552ed8616eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:00.240670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92d496-011e-4814-a6b2-b552ed8616eb admin admin] PATH_INFO: `/v3/projects/b8fd58bb440d40bda9dc45610863470b/users/14f8aea0e3574909801c2e4c8ad0e698/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:00.255548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92d496-011e-4814-a6b2-b552ed8616eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8fd58bb440d40bda9dc45610863470b, user_id=14f8aea0e3574909801c2e4c8ad0e698, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:00.258355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92d496-011e-4814-a6b2-b552ed8616eb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:00.355593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 853/1703] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:00 2022] PUT /identity/v3/projects/b8fd58bb440d40bda9dc45610863470b/users/14f8aea0e3574909801c2e4c8ad0e698/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:00.366860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84051d-2b7b-4061-8b08-30bc3e929e04 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:00.367286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84051d-2b7b-4061-8b08-30bc3e929e04 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:00.367666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84051d-2b7b-4061-8b08-30bc3e929e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:00.406443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2b84051d-2b7b-4061-8b08-30bc3e929e04 None None] MFA Rules not processed for user `14f8aea0e3574909801c2e4c8ad0e698`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:00.408005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2b84051d-2b7b-4061-8b08-30bc3e929e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:00.448043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:00.577811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 851/1704] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:00 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:00.580358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92d496-011e-4814-a6b2-b552ed8616eb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:00.777793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92d496-011e-4814-a6b2-b552ed8616eb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:00.788239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d3224-e4fa-40d9-9e44-04a3ca55f92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:00.789416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d3224-e4fa-40d9-9e44-04a3ca55f92a admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:00.789839 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d3224-e4fa-40d9-9e44-04a3ca55f92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:00.790410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d3224-e4fa-40d9-9e44-04a3ca55f92a admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:00.792315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d3224-e4fa-40d9-9e44-04a3ca55f92a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:00.794829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d3224-e4fa-40d9-9e44-04a3ca55f92a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:00.895083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84051d-2b7b-4061-8b08-30bc3e929e04 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:00.908018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2b84051d-2b7b-4061-8b08-30bc3e929e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:00.942170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 854/1705] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:00 2022] POST /identity/v3/projects => generated 409 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:00.944541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d3224-e4fa-40d9-9e44-04a3ca55f92a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:00.966710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84051d-2b7b-4061-8b08-30bc3e929e04 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:01.094533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d3224-e4fa-40d9-9e44-04a3ca55f92a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:01.102284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fb7fe-c96f-4db7-8d0e-0afcf6387681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:01.103273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fb7fe-c96f-4db7-8d0e-0afcf6387681 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:01.103729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fb7fe-c96f-4db7-8d0e-0afcf6387681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:01.104324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fb7fe-c96f-4db7-8d0e-0afcf6387681 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:01.105384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fb7fe-c96f-4db7-8d0e-0afcf6387681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:01.108008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fb7fe-c96f-4db7-8d0e-0afcf6387681 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:01.160921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:01.161574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:01.162238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:01.164270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] PATH_INFO: `/v3/users/5baa38db92664d50a96672b9f57f4c3b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:01.165237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5baa38db92664d50a96672b9f57f4c3b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:01.167591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:01.247779 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 855/1706] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:00 2022] POST /identity/v3/users => generated 311 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:01.254102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fb7fe-c96f-4db7-8d0e-0afcf6387681 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:01.294926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fb7fe-c96f-4db7-8d0e-0afcf6387681 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:01.337579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0d08c-7ab7-4992-b738-a76a0d198a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:01.341011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0d08c-7ab7-4992-b738-a76a0d198a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:01.341311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0d08c-7ab7-4992-b738-a76a0d198a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:01.342888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0d08c-7ab7-4992-b738-a76a0d198a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:01.343595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0d08c-7ab7-4992-b738-a76a0d198a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:01.349634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0d08c-7ab7-4992-b738-a76a0d198a00 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:01.406725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 856/1707] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:01.411515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0d08c-7ab7-4992-b738-a76a0d198a00 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:01.457383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0d08c-7ab7-4992-b738-a76a0d198a00 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:01.467179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5baa38db92664d50a96672b9f57f4c3b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:01.470719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5baa38db92664d50a96672b9f57f4c3b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:01.472800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5baa38db92664d50a96672b9f57f4c3b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:01.484745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945031e0-4e4b-4a4b-be14-919fd8d34734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:01.487205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945031e0-4e4b-4a4b-be14-919fd8d34734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:01.487205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945031e0-4e4b-4a4b-be14-919fd8d34734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:01.490540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945031e0-4e4b-4a4b-be14-919fd8d34734 admin admin] PATH_INFO: `/v3/projects/55f37a172b6c49048890033f36548342/users/38d631eaee7c42538876cb4911598119/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:01.551276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945031e0-4e4b-4a4b-be14-919fd8d34734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f37a172b6c49048890033f36548342, user_id=38d631eaee7c42538876cb4911598119, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:01.554034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945031e0-4e4b-4a4b-be14-919fd8d34734 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:01.589464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5baa38db92664d50a96672b9f57f4c3b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:01.628594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e2c2a4f0-5f73-45fc-980a-835937b10bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5baa38db92664d50a96672b9f57f4c3b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:01.676818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 852/1708] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:00 2022] DELETE /identity/v3/users/5baa38db92664d50a96672b9f57f4c3b => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:01.678543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d220fa1-01ec-4f84-81ef-c3306d0b744a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:01.678753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d220fa1-01ec-4f84-81ef-c3306d0b744a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:01.679036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d220fa1-01ec-4f84-81ef-c3306d0b744a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:01.703995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 857/1709] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:01 2022] PUT /identity/v3/projects/55f37a172b6c49048890033f36548342/users/38d631eaee7c42538876cb4911598119/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:01.707035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058eb65-e0f4-4bb0-b335-828c76cd7013 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:01.709806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058eb65-e0f4-4bb0-b335-828c76cd7013 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:01.710114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058eb65-e0f4-4bb0-b335-828c76cd7013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:01.747248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:01.762653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3058eb65-e0f4-4bb0-b335-828c76cd7013 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:01.768731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3058eb65-e0f4-4bb0-b335-828c76cd7013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:01.829112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9d220fa1-01ec-4f84-81ef-c3306d0b744a None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:01.830983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9d220fa1-01ec-4f84-81ef-c3306d0b744a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:01.850758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:01.922385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 858/1710] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:01.926512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058eb65-e0f4-4bb0-b335-828c76cd7013 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:01.930908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3058eb65-e0f4-4bb0-b335-828c76cd7013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:01.952722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058eb65-e0f4-4bb0-b335-828c76cd7013 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:01.962314 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 853/1711] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:25:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:01.964646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d220fa1-01ec-4f84-81ef-c3306d0b744a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:01.966924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9d220fa1-01ec-4f84-81ef-c3306d0b744a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:01.967955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3472e5-3bff-4ed9-a74d-29497b6965f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:01.968588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3472e5-3bff-4ed9-a74d-29497b6965f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:01.968860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3472e5-3bff-4ed9-a74d-29497b6965f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:01.969392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3472e5-3bff-4ed9-a74d-29497b6965f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:01.970231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3472e5-3bff-4ed9-a74d-29497b6965f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:01.974537 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3472e5-3bff-4ed9-a74d-29497b6965f6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:01.997631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 859/1712] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:02.003045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3472e5-3bff-4ed9-a74d-29497b6965f6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:02.027799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3472e5-3bff-4ed9-a74d-29497b6965f6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:02.034292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d220fa1-01ec-4f84-81ef-c3306d0b744a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:02.038848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d17fe-2a6f-4440-83b8-5135f1adff1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:02.039807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d17fe-2a6f-4440-83b8-5135f1adff1e service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:02.040762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d17fe-2a6f-4440-83b8-5135f1adff1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:02.041290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d17fe-2a6f-4440-83b8-5135f1adff1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:02.042560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c7cd-a257-4d61-8da9-214b8b26b3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:02.043252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c7cd-a257-4d61-8da9-214b8b26b3be admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:02.043517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c7cd-a257-4d61-8da9-214b8b26b3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:02.044288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c7cd-a257-4d61-8da9-214b8b26b3be admin admin] PATH_INFO: `/v3/projects/55f37a172b6c49048890033f36548342/users/38d631eaee7c42538876cb4911598119/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:02.048250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281c7cd-a257-4d61-8da9-214b8b26b3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f37a172b6c49048890033f36548342, user_id=38d631eaee7c42538876cb4911598119, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:02.055897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281c7cd-a257-4d61-8da9-214b8b26b3be admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:02.060125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d17fe-2a6f-4440-83b8-5135f1adff1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:02.062130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d17fe-2a6f-4440-83b8-5135f1adff1e service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:02.076180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 854/1713] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:02.089503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 860/1714] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:02 2022] PUT /identity/v3/projects/55f37a172b6c49048890033f36548342/users/38d631eaee7c42538876cb4911598119/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:02.101340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a706b-3901-4c03-8416-a1e4910ac129 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:02.102025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a706b-3901-4c03-8416-a1e4910ac129 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:02.102487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a706b-3901-4c03-8416-a1e4910ac129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:02.174964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-090a706b-3901-4c03-8416-a1e4910ac129 None None] MFA Rules not processed for user `38d631eaee7c42538876cb4911598119`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:02.176837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-090a706b-3901-4c03-8416-a1e4910ac129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:02.350107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 855/1715] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:02 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:02.418038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c7cd-a257-4d61-8da9-214b8b26b3be admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:02.419917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4281c7cd-a257-4d61-8da9-214b8b26b3be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:02.427975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a706b-3901-4c03-8416-a1e4910ac129 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:02.430200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-090a706b-3901-4c03-8416-a1e4910ac129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:02.577178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a706b-3901-4c03-8416-a1e4910ac129 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:02.595206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:02.597079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee4c9e-0236-4fa9-88b5-d9c8b987d035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:02.597756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee4c9e-0236-4fa9-88b5-d9c8b987d035 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:02.598066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee4c9e-0236-4fa9-88b5-d9c8b987d035 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:02.598340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee4c9e-0236-4fa9-88b5-d9c8b987d035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:02.628851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee4c9e-0236-4fa9-88b5-d9c8b987d035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:02.630357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee4c9e-0236-4fa9-88b5-d9c8b987d035 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:02.655346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c7cd-a257-4d61-8da9-214b8b26b3be admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:02.664145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 856/1716] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:02 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:02.667433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3d2b0-b9ee-4654-85db-cec9e16e3b1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:02.667890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3d2b0-b9ee-4654-85db-cec9e16e3b1b service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:02.668175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3d2b0-b9ee-4654-85db-cec9e16e3b1b service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:02.668454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3d2b0-b9ee-4654-85db-cec9e16e3b1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:02.792939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3d2b0-b9ee-4654-85db-cec9e16e3b1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:02.794237 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3d2b0-b9ee-4654-85db-cec9e16e3b1b service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:02.855167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 861/1717] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:25:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:25:02.901060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430bb88-d1dd-4e75-a1e3-3ac3a9c9de8c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:02.901526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430bb88-d1dd-4e75-a1e3-3ac3a9c9de8c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:02.907107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430bb88-d1dd-4e75-a1e3-3ac3a9c9de8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:02.962980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c430bb88-d1dd-4e75-a1e3-3ac3a9c9de8c None None] MFA Rules not processed for user `b0145242ef784168bada8348ccd95f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:02.968576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c430bb88-d1dd-4e75-a1e3-3ac3a9c9de8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:03.205531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 857/1718] 10.222.0.102 () {58 vars in 1016 bytes} [Wed Aug 24 11:25:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:03.331536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3d2b0-b9ee-4654-85db-cec9e16e3b1b service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:03.337733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:03.361468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3d2b0-b9ee-4654-85db-cec9e16e3b1b service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:03.374817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41581792-8921-4602-9e03-264806ed6b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:03.376673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41581792-8921-4602-9e03-264806ed6b92 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:03.377625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41581792-8921-4602-9e03-264806ed6b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:03.378275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41581792-8921-4602-9e03-264806ed6b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:03.405508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41581792-8921-4602-9e03-264806ed6b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:03.412441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41581792-8921-4602-9e03-264806ed6b92 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:03.453769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 862/1719] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:03.621237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430bb88-d1dd-4e75-a1e3-3ac3a9c9de8c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:03.651951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430bb88-d1dd-4e75-a1e3-3ac3a9c9de8c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:03.668999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:03.669931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:03.670399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:03.670820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:03.681710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:03.751159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41581792-8921-4602-9e03-264806ed6b92 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:03.755480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-41581792-8921-4602-9e03-264806ed6b92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:03.852267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:03.854748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:03.867912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 858/1720] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:03 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:03.925999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41581792-8921-4602-9e03-264806ed6b92 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:03.932139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:03.933029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:03.933539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:03.943184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] PATH_INFO: `/v3/users/64db87b93e5544f1a6b8167e2003e200` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:03.944823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] RBAC: Authorizing `identity:delete_user(user_id=64db87b93e5544f1a6b8167e2003e200)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:03.947042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:03.976120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:03.978470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:04.022320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2845125-e982-4f0e-ac43-6b70837b4761 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:04.029209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:04.031442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:04.031442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:04.031442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] PATH_INFO: `/v3/projects/4261e17a79fa4878a2fd3da019e13b5b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:04.032604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4261e17a79fa4878a2fd3da019e13b5b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:04.035912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:04.228859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4261e17a79fa4878a2fd3da019e13b5b', 'request_id': 'req-22db7961-3916-41e6-beb5-f9a2d1a1f78c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.237374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4261e17a79fa4878a2fd3da019e13b5b', 'request_id': 'req-22db7961-3916-41e6-beb5-f9a2d1a1f78c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.250313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4261e17a79fa4878a2fd3da019e13b5b', 'request_id': 'req-22db7961-3916-41e6-beb5-f9a2d1a1f78c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.257205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64db87b93e5544f1a6b8167e2003e200'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.259374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64db87b93e5544f1a6b8167e2003e200'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.260697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64db87b93e5544f1a6b8167e2003e200'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.306034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64db87b93e5544f1a6b8167e2003e200'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.343683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] The token cache is being invalidate because project 4261e17a79fa4878a2fd3da019e13b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:04.344157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.345656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-22db7961-3916-41e6-beb5-f9a2d1a1f78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.353168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 859/1721] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:03 2022] DELETE /identity/v3/projects/4261e17a79fa4878a2fd3da019e13b5b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:04.357275 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c7970-dfef-4e3d-824e-fcee626af319 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:04.358448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c7970-dfef-4e3d-824e-fcee626af319 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:04.361533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d2173e0d-6caa-4dcd-876a-d17929d7f38e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64db87b93e5544f1a6b8167e2003e200'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:04.363296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c7970-dfef-4e3d-824e-fcee626af319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:04.440453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cf1c7970-dfef-4e3d-824e-fcee626af319 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:04.441811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cf1c7970-dfef-4e3d-824e-fcee626af319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:04.462364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:04.526438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 863/1722] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:03 2022] DELETE /identity/v3/users/64db87b93e5544f1a6b8167e2003e200 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:04.531522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6dbf5-1d66-4237-9414-80be30e98c38 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:04.531825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6dbf5-1d66-4237-9414-80be30e98c38 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:04.532076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6dbf5-1d66-4237-9414-80be30e98c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:04.620265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0cf6dbf5-1d66-4237-9414-80be30e98c38 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:04.620265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0cf6dbf5-1d66-4237-9414-80be30e98c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:04.630262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 860/1723] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:04.642137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47199d-5f19-42d3-b2cf-ec9cf7e7b412 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:04.648392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47199d-5f19-42d3-b2cf-ec9cf7e7b412 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:04.648392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47199d-5f19-42d3-b2cf-ec9cf7e7b412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:04.739838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ba47199d-5f19-42d3-b2cf-ec9cf7e7b412 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:04.746756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ba47199d-5f19-42d3-b2cf-ec9cf7e7b412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:04.821288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 864/1724] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:04.858333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6dbf5-1d66-4237-9414-80be30e98c38 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:04.858333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0cf6dbf5-1d66-4237-9414-80be30e98c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:04.881527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:04.914989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 861/1725] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:04.921835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47199d-5f19-42d3-b2cf-ec9cf7e7b412 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:04.960227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47199d-5f19-42d3-b2cf-ec9cf7e7b412 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:04.978724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a29dfd-edd1-4207-86f9-0383c85f383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:04.980042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a29dfd-edd1-4207-86f9-0383c85f383d admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:04.980736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a29dfd-edd1-4207-86f9-0383c85f383d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:04.981241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a29dfd-edd1-4207-86f9-0383c85f383d admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:04.982255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a29dfd-edd1-4207-86f9-0383c85f383d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:25:04.986599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a29dfd-edd1-4207-86f9-0383c85f383d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:04.997106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a29dfd-edd1-4207-86f9-0383c85f383d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:05.022566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6dbf5-1d66-4237-9414-80be30e98c38 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:05.034643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b282d8-98e9-44d8-a1ca-689e9f4eb21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:05.035414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b282d8-98e9-44d8-a1ca-689e9f4eb21c service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:05.036179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b282d8-98e9-44d8-a1ca-689e9f4eb21c service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:05.037689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b282d8-98e9-44d8-a1ca-689e9f4eb21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:05.039122 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 862/1726] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:25:05.043784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a29dfd-edd1-4207-86f9-0383c85f383d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:05.055101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a29dfd-edd1-4207-86f9-0383c85f383d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:05.064697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b282d8-98e9-44d8-a1ca-689e9f4eb21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:05.066380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b282d8-98e9-44d8-a1ca-689e9f4eb21c service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:05.080463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 865/1727] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:05.083877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89274c74-9124-42ed-8cce-4b8b37832589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:05.084776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89274c74-9124-42ed-8cce-4b8b37832589 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:05.085131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89274c74-9124-42ed-8cce-4b8b37832589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:05.085742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89274c74-9124-42ed-8cce-4b8b37832589 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:05.086457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89274c74-9124-42ed-8cce-4b8b37832589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:05.088719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89274c74-9124-42ed-8cce-4b8b37832589 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:05.254932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 863/1728] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:05 2022] POST /identity/v3/projects => generated 403 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:05.259192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b282d8-98e9-44d8-a1ca-689e9f4eb21c service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:05.297144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b282d8-98e9-44d8-a1ca-689e9f4eb21c service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:05.513554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f31bf-1c04-4af6-8e5a-08a72b0a7f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:05.514556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f31bf-1c04-4af6-8e5a-08a72b0a7f64 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:05.514888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f31bf-1c04-4af6-8e5a-08a72b0a7f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:05.524583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f31bf-1c04-4af6-8e5a-08a72b0a7f64 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:05.524583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f31bf-1c04-4af6-8e5a-08a72b0a7f64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:05.524583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f31bf-1c04-4af6-8e5a-08a72b0a7f64 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:05.658896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 866/1729] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:05 2022] POST /identity/v3/users => generated 302 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:05.676833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89274c74-9124-42ed-8cce-4b8b37832589 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:05.700890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89274c74-9124-42ed-8cce-4b8b37832589 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:05.727342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00870f-f93b-4552-afa0-932592f17049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:05.732112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00870f-f93b-4552-afa0-932592f17049 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:05.732112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00870f-f93b-4552-afa0-932592f17049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:05.732112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00870f-f93b-4552-afa0-932592f17049 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:05.732112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00870f-f93b-4552-afa0-932592f17049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:05.740209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00870f-f93b-4552-afa0-932592f17049 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:05.777384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 864/1730] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:05.779139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f31bf-1c04-4af6-8e5a-08a72b0a7f64 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:05.810219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f31bf-1c04-4af6-8e5a-08a72b0a7f64 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:05.823461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb1371-b8d6-4e3f-8fef-557fcb80efe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:05.824099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb1371-b8d6-4e3f-8fef-557fcb80efe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:05.824382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb1371-b8d6-4e3f-8fef-557fcb80efe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:05.824706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb1371-b8d6-4e3f-8fef-557fcb80efe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:05.825471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb1371-b8d6-4e3f-8fef-557fcb80efe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:05.828872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb1371-b8d6-4e3f-8fef-557fcb80efe4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:05.857278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00870f-f93b-4552-afa0-932592f17049 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:05.857667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 867/1731] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:05.880270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00870f-f93b-4552-afa0-932592f17049 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:05.898196 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f579be-fa50-4d53-ac36-31582ebdc933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:05.899332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f579be-fa50-4d53-ac36-31582ebdc933 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:05.899573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f579be-fa50-4d53-ac36-31582ebdc933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:05.899856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f579be-fa50-4d53-ac36-31582ebdc933 admin admin] PATH_INFO: `/v3/projects/794f120571ed40e1bd6ddf4011bdd849/users/12a6a635daff452a99c82cb551c3cecc/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:05.934443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f579be-fa50-4d53-ac36-31582ebdc933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=794f120571ed40e1bd6ddf4011bdd849, user_id=12a6a635daff452a99c82cb551c3cecc, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:05.939692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f579be-fa50-4d53-ac36-31582ebdc933 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:06.122728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 865/1732] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:05 2022] PUT /identity/v3/projects/794f120571ed40e1bd6ddf4011bdd849/users/12a6a635daff452a99c82cb551c3cecc/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:06.125685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e72d6b-b0b9-42bd-9fc8-ce5654cd23a2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:06.126211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e72d6b-b0b9-42bd-9fc8-ce5654cd23a2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:06.126535 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e72d6b-b0b9-42bd-9fc8-ce5654cd23a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:06.219977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-81e72d6b-b0b9-42bd-9fc8-ce5654cd23a2 None None] MFA Rules not processed for user `12a6a635daff452a99c82cb551c3cecc`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:06.224103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-81e72d6b-b0b9-42bd-9fc8-ce5654cd23a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:06.431559 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 868/1733] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:06.438382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f579be-fa50-4d53-ac36-31582ebdc933 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:06.468829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e72d6b-b0b9-42bd-9fc8-ce5654cd23a2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:06.471477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-81e72d6b-b0b9-42bd-9fc8-ce5654cd23a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:06.509793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:06.533999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:06.547913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f579be-fa50-4d53-ac36-31582ebdc933 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:06.547913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acfa1a-0060-4be2-9fd8-cb2f11e397c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:06.548786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acfa1a-0060-4be2-9fd8-cb2f11e397c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:06.549253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acfa1a-0060-4be2-9fd8-cb2f11e397c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:06.550089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acfa1a-0060-4be2-9fd8-cb2f11e397c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:06.551176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acfa1a-0060-4be2-9fd8-cb2f11e397c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:06.554445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acfa1a-0060-4be2-9fd8-cb2f11e397c0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:06.624771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e72d6b-b0b9-42bd-9fc8-ce5654cd23a2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:06.631352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:06.652176 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:06.652896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:06.655696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] PATH_INFO: `/v3/projects/6546cd2088ee4c29890d6841ac1a2686` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:06.661447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6546cd2088ee4c29890d6841ac1a2686)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:06.666340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:06.763608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 866/1734] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:06 2022] POST /identity/v3/projects => generated 403 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:06.770446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acfa1a-0060-4be2-9fd8-cb2f11e397c0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:06.896129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acfa1a-0060-4be2-9fd8-cb2f11e397c0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:06.921747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6546cd2088ee4c29890d6841ac1a2686', 'request_id': 'req-7207c6f0-782e-4dde-b44f-34b72a2ef309'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:06.940278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6546cd2088ee4c29890d6841ac1a2686', 'request_id': 'req-7207c6f0-782e-4dde-b44f-34b72a2ef309'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:07.002388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688dd4a-e98d-493e-90ce-857b1cc3c4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:07.003073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688dd4a-e98d-493e-90ce-857b1cc3c4dc admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.003358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688dd4a-e98d-493e-90ce-857b1cc3c4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.003629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688dd4a-e98d-493e-90ce-857b1cc3c4dc admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.004517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688dd4a-e98d-493e-90ce-857b1cc3c4dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:07.006520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688dd4a-e98d-493e-90ce-857b1cc3c4dc admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:07.051467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6546cd2088ee4c29890d6841ac1a2686', 'request_id': 'req-7207c6f0-782e-4dde-b44f-34b72a2ef309'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:07.066339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] The token cache is being invalidate because project 6546cd2088ee4c29890d6841ac1a2686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:07.066636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:07.067835 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7207c6f0-782e-4dde-b44f-34b72a2ef309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:07.069753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 869/1735] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:06 2022] DELETE /identity/v3/projects/6546cd2088ee4c29890d6841ac1a2686 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:07.071217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9cae7-6fe6-42a0-b1fc-b4d3006abb53 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.071485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9cae7-6fe6-42a0-b1fc-b4d3006abb53 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.071774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9cae7-6fe6-42a0-b1fc-b4d3006abb53 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.073529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 870/1736] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:07.075366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ecbec-a4ba-473b-8bfa-186829722d8d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.076003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ecbec-a4ba-473b-8bfa-186829722d8d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.077596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ecbec-a4ba-473b-8bfa-186829722d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.122131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b18ecbec-a4ba-473b-8bfa-186829722d8d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:07.123921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b18ecbec-a4ba-473b-8bfa-186829722d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:07.144296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 871/1737] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:07.147122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c2529-3378-4608-aa3c-e1f22bab69eb None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.147637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c2529-3378-4608-aa3c-e1f22bab69eb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.148280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c2529-3378-4608-aa3c-e1f22bab69eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.200804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1c6c2529-3378-4608-aa3c-e1f22bab69eb None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:07.200804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1c6c2529-3378-4608-aa3c-e1f22bab69eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:07.231751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 867/1738] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:06 2022] POST /identity/v3/users => generated 308 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:07.233856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb65c62-3aa5-4642-b90a-4f5b32849b41 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.236118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb65c62-3aa5-4642-b90a-4f5b32849b41 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.236118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb65c62-3aa5-4642-b90a-4f5b32849b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.308870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-afb65c62-3aa5-4642-b90a-4f5b32849b41 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:07.310536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-afb65c62-3aa5-4642-b90a-4f5b32849b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:07.352078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 868/1739] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:07.352078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb65c62-3aa5-4642-b90a-4f5b32849b41 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:07.358021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-afb65c62-3aa5-4642-b90a-4f5b32849b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:07.374449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb65c62-3aa5-4642-b90a-4f5b32849b41 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:07.380763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1acbe2-8b00-4635-85ab-f0a657dad723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:07.391675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1acbe2-8b00-4635-85ab-f0a657dad723 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.391675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1acbe2-8b00-4635-85ab-f0a657dad723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.391675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1acbe2-8b00-4635-85ab-f0a657dad723 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.391675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1acbe2-8b00-4635-85ab-f0a657dad723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:07.391675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1acbe2-8b00-4635-85ab-f0a657dad723 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:07.392237 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 872/1740] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:07.392355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c2529-3378-4608-aa3c-e1f22bab69eb None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:07.399219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 869/1741] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:07.403150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1acbe2-8b00-4635-85ab-f0a657dad723 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:07.427336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1acbe2-8b00-4635-85ab-f0a657dad723 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:07.431703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c2529-3378-4608-aa3c-e1f22bab69eb None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:07.436364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5bb78-1739-4427-91a2-32bfcb31d255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:07.437411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5bb78-1739-4427-91a2-32bfcb31d255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.437546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5bb78-1739-4427-91a2-32bfcb31d255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.438002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5bb78-1739-4427-91a2-32bfcb31d255 admin admin] PATH_INFO: `/v3/projects/d15f287e716e4b1f89bf3dcdb9fdaf1a/users/65fbcb6af26b4cfd948cc606810b76c7/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.446040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fa1b3-1b39-4ed2-90f4-0581292c16f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:07.448402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fa1b3-1b39-4ed2-90f4-0581292c16f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.448793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fa1b3-1b39-4ed2-90f4-0581292c16f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.449065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fa1b3-1b39-4ed2-90f4-0581292c16f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.454010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fa1b3-1b39-4ed2-90f4-0581292c16f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:25:07.459408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fa1b3-1b39-4ed2-90f4-0581292c16f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:07.459408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fa1b3-1b39-4ed2-90f4-0581292c16f9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:07.473613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5bb78-1739-4427-91a2-32bfcb31d255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15f287e716e4b1f89bf3dcdb9fdaf1a, user_id=65fbcb6af26b4cfd948cc606810b76c7, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:07.476918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5bb78-1739-4427-91a2-32bfcb31d255 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:07.492290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 873/1742] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:25:07.506887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fa1b3-1b39-4ed2-90f4-0581292c16f9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:07.557588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fa1b3-1b39-4ed2-90f4-0581292c16f9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:07.584447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcba822-a512-4525-ac7b-74147728641e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:07.585080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcba822-a512-4525-ac7b-74147728641e admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.586967 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcba822-a512-4525-ac7b-74147728641e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.587261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcba822-a512-4525-ac7b-74147728641e admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.587915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcba822-a512-4525-ac7b-74147728641e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:07.591673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 870/1743] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:07 2022] PUT /identity/v3/projects/d15f287e716e4b1f89bf3dcdb9fdaf1a/users/65fbcb6af26b4cfd948cc606810b76c7/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:07.591785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcba822-a512-4525-ac7b-74147728641e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:07.591861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5bb78-1739-4427-91a2-32bfcb31d255 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:07.594473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-beb5bb78-1739-4427-91a2-32bfcb31d255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:07.758406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 874/1744] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:07 2022] POST /identity/v3/projects => generated 373 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:07.759371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcba822-a512-4525-ac7b-74147728641e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:07.811881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:07.811881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5bb78-1739-4427-91a2-32bfcb31d255 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:07.872986 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:07.950795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcba822-a512-4525-ac7b-74147728641e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:07.960908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f4dd6-a488-415c-a42c-83a785a847b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:07.964146 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f4dd6-a488-415c-a42c-83a785a847b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.964436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f4dd6-a488-415c-a42c-83a785a847b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.964712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f4dd6-a488-415c-a42c-83a785a847b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.965487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f4dd6-a488-415c-a42c-83a785a847b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:07.967047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f4dd6-a488-415c-a42c-83a785a847b3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:07.992687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9c333-9f38-4dcc-8ad1-8a0ecd26eb19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:07.993377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9c333-9f38-4dcc-8ad1-8a0ecd26eb19 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:07.993650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9c333-9f38-4dcc-8ad1-8a0ecd26eb19 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:07.994231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9c333-9f38-4dcc-8ad1-8a0ecd26eb19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:07.994333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 875/1745] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:07.995607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f4dd6-a488-415c-a42c-83a785a847b3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:08.008996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9c333-9f38-4dcc-8ad1-8a0ecd26eb19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:08.011519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9c333-9f38-4dcc-8ad1-8a0ecd26eb19 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:08.023740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f4dd6-a488-415c-a42c-83a785a847b3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:08.028383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 871/1746] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:08.029419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d9db9-26d5-4756-935a-6cdeffd321fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:08.030570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d9db9-26d5-4756-935a-6cdeffd321fc admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.030857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d9db9-26d5-4756-935a-6cdeffd321fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.031239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d9db9-26d5-4756-935a-6cdeffd321fc admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.031956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9c333-9f38-4dcc-8ad1-8a0ecd26eb19 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:08.032064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d9db9-26d5-4756-935a-6cdeffd321fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:08.037519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d9db9-26d5-4756-935a-6cdeffd321fc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:08.049031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9c333-9f38-4dcc-8ad1-8a0ecd26eb19 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:08.055504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730636d0-5d76-4ef6-92ba-84dfe05cc0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:08.056362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730636d0-5d76-4ef6-92ba-84dfe05cc0f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.056739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730636d0-5d76-4ef6-92ba-84dfe05cc0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.057176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730636d0-5d76-4ef6-92ba-84dfe05cc0f0 admin admin] PATH_INFO: `/v3/projects/d15f287e716e4b1f89bf3dcdb9fdaf1a/users/65fbcb6af26b4cfd948cc606810b76c7/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.059478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730636d0-5d76-4ef6-92ba-84dfe05cc0f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15f287e716e4b1f89bf3dcdb9fdaf1a, user_id=65fbcb6af26b4cfd948cc606810b76c7, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:08.066487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730636d0-5d76-4ef6-92ba-84dfe05cc0f0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:08.081830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 872/1747] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:08 2022] PUT /identity/v3/projects/d15f287e716e4b1f89bf3dcdb9fdaf1a/users/65fbcb6af26b4cfd948cc606810b76c7/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:08.094525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde2d40-cc7f-4c62-b419-29fc01bc0ef6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.095856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde2d40-cc7f-4c62-b419-29fc01bc0ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.096285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde2d40-cc7f-4c62-b419-29fc01bc0ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.126052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 876/1748] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:07 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:08.131885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d9db9-26d5-4756-935a-6cdeffd321fc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:08.146221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d9db9-26d5-4756-935a-6cdeffd321fc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:08.195192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fdde2d40-cc7f-4c62-b419-29fc01bc0ef6 None None] MFA Rules not processed for user `65fbcb6af26b4cfd948cc606810b76c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:08.197041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fdde2d40-cc7f-4c62-b419-29fc01bc0ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:08.302417 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6ed20-9abc-446b-b417-1cb76309029f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:08.303492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6ed20-9abc-446b-b417-1cb76309029f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.303987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6ed20-9abc-446b-b417-1cb76309029f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.304507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6ed20-9abc-446b-b417-1cb76309029f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.305374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6ed20-9abc-446b-b417-1cb76309029f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:08.308662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6ed20-9abc-446b-b417-1cb76309029f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:08.334014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 877/1749] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:08.349497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4b032-6c58-4f4a-a668-5ee550f4a71c None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.350212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4b032-6c58-4f4a-a668-5ee550f4a71c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.350866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4b032-6c58-4f4a-a668-5ee550f4a71c None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.353389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 878/1750] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:25:08 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:08.361708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4b032-6c58-4f4a-a668-5ee550f4a71c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:08.383042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4b032-6c58-4f4a-a668-5ee550f4a71c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:08.391799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5efea-9d1c-456b-a53b-a60af34a9f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:08.392958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5efea-9d1c-456b-a53b-a60af34a9f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.394163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5efea-9d1c-456b-a53b-a60af34a9f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.394723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5efea-9d1c-456b-a53b-a60af34a9f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.396205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5efea-9d1c-456b-a53b-a60af34a9f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:08.398400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5efea-9d1c-456b-a53b-a60af34a9f6c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:08.422665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 879/1751] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:08.427380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5efea-9d1c-456b-a53b-a60af34a9f6c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:08.431531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 873/1752] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:08 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:08.447752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5efea-9d1c-456b-a53b-a60af34a9f6c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:08.469710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa0802-c254-4381-9984-c1ffa4c8f879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:08.470949 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa0802-c254-4381-9984-c1ffa4c8f879 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.471457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa0802-c254-4381-9984-c1ffa4c8f879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.472106 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa0802-c254-4381-9984-c1ffa4c8f879 admin admin] PATH_INFO: `/v3/projects/7e11f56f3d1d4ed1874d78e5e215e213/users/a5ab3b8e84fa429a91383f24214a88ce/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.494604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141c912-d0d3-42ec-9232-f17695ff4680 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.495276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141c912-d0d3-42ec-9232-f17695ff4680 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.496086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141c912-d0d3-42ec-9232-f17695ff4680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.528016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa0802-c254-4381-9984-c1ffa4c8f879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e11f56f3d1d4ed1874d78e5e215e213, user_id=a5ab3b8e84fa429a91383f24214a88ce, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:08.533447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa0802-c254-4381-9984-c1ffa4c8f879 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:08.543965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0141c912-d0d3-42ec-9232-f17695ff4680 None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:08.547303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0141c912-d0d3-42ec-9232-f17695ff4680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:08.646457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 880/1753] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:08 2022] PUT /identity/v3/projects/7e11f56f3d1d4ed1874d78e5e215e213/users/a5ab3b8e84fa429a91383f24214a88ce/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:08.646457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa0802-c254-4381-9984-c1ffa4c8f879 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:08.646457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d5aa0802-c254-4381-9984-c1ffa4c8f879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:08.707892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 874/1754] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:25:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:08.709453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bec13-a0ad-4a28-ae10-174c560d99b9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.709795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bec13-a0ad-4a28-ae10-174c560d99b9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.710885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bec13-a0ad-4a28-ae10-174c560d99b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.799665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa0802-c254-4381-9984-c1ffa4c8f879 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:08.807698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21c39c-3e8f-4836-b8d5-4c03aef6654d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:08.810498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21c39c-3e8f-4836-b8d5-4c03aef6654d service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:08.810780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21c39c-3e8f-4836-b8d5-4c03aef6654d service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:08.811227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21c39c-3e8f-4836-b8d5-4c03aef6654d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:08.831687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e83bec13-a0ad-4a28-ae10-174c560d99b9 None None] MFA Rules not processed for user `a5ab3b8e84fa429a91383f24214a88ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:08.834991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e83bec13-a0ad-4a28-ae10-174c560d99b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:08.933297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:09.047787 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21c39c-3e8f-4836-b8d5-4c03aef6654d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:09.053509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21c39c-3e8f-4836-b8d5-4c03aef6654d service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:09.080935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 881/1755] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:08 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:09.083428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21c39c-3e8f-4836-b8d5-4c03aef6654d service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:09.084971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 875/1756] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:09.089339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bec13-a0ad-4a28-ae10-174c560d99b9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:09.091868 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ef21c39c-3e8f-4836-b8d5-4c03aef6654d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:09.221359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bec13-a0ad-4a28-ae10-174c560d99b9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:09.243898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851d28-ff75-4910-842f-59e0ebfa8248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:09.244662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851d28-ff75-4910-842f-59e0ebfa8248 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:09.245084 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851d28-ff75-4910-842f-59e0ebfa8248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:09.245425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851d28-ff75-4910-842f-59e0ebfa8248 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:09.246298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70851d28-ff75-4910-842f-59e0ebfa8248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:09.248466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70851d28-ff75-4910-842f-59e0ebfa8248 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:09.311185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21c39c-3e8f-4836-b8d5-4c03aef6654d service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:09.320799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903a72b-12bf-4176-889a-bf32c7f66170 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:09.321447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903a72b-12bf-4176-889a-bf32c7f66170 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:09.321724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903a72b-12bf-4176-889a-bf32c7f66170 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:09.322127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903a72b-12bf-4176-889a-bf32c7f66170 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:09.348421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903a72b-12bf-4176-889a-bf32c7f66170 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:09.350159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903a72b-12bf-4176-889a-bf32c7f66170 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:09.372067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 882/1757] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:09.418812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 876/1758] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:09 2022] POST /identity/v3/projects => generated 373 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:09.422429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903a72b-12bf-4176-889a-bf32c7f66170 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:09.447458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903a72b-12bf-4176-889a-bf32c7f66170 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:09.501641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851d28-ff75-4910-842f-59e0ebfa8248 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:09.546126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e2968-6562-4827-b860-a77cd6c70b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:09.546524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e2968-6562-4827-b860-a77cd6c70b76 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:09.553416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e2968-6562-4827-b860-a77cd6c70b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:09.553416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e2968-6562-4827-b860-a77cd6c70b76 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:09.553416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e2968-6562-4827-b860-a77cd6c70b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:09.553416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e2968-6562-4827-b860-a77cd6c70b76 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:09.659433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 883/1759] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:09 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:09.661029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e2968-6562-4827-b860-a77cd6c70b76 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:09.679448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851d28-ff75-4910-842f-59e0ebfa8248 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:09.683725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:09.684405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:09.685370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:09.697700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e2968-6562-4827-b860-a77cd6c70b76 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:09.706767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] PATH_INFO: `/v3/users/12a6a635daff452a99c82cb551c3cecc` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:09.708138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a6a635daff452a99c82cb551c3cecc)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:09.710301 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:09.710757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04870a-b698-485d-a3d6-c2f38b05b796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:09.711395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04870a-b698-485d-a3d6-c2f38b05b796 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:09.711730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04870a-b698-485d-a3d6-c2f38b05b796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:09.711943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04870a-b698-485d-a3d6-c2f38b05b796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:09.712960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04870a-b698-485d-a3d6-c2f38b05b796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:09.714500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04870a-b698-485d-a3d6-c2f38b05b796 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:09.730840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:09.742428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 884/1760] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:09.746790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04870a-b698-485d-a3d6-c2f38b05b796 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:09.759549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04870a-b698-485d-a3d6-c2f38b05b796 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:09.766377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e4629-4aa9-4301-bac0-f4856cab526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:09.767173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e4629-4aa9-4301-bac0-f4856cab526f admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:09.767467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e4629-4aa9-4301-bac0-f4856cab526f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:09.767790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e4629-4aa9-4301-bac0-f4856cab526f admin admin] PATH_INFO: `/v3/projects/eae17eee05c14fc19fe8538df9e7080f/users/c8c5987f72594a79adddec3c5eccc0ab/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:09.799694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e4629-4aa9-4301-bac0-f4856cab526f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae17eee05c14fc19fe8538df9e7080f, user_id=c8c5987f72594a79adddec3c5eccc0ab, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:09.805219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e4629-4aa9-4301-bac0-f4856cab526f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:09.931959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 885/1761] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:09 2022] PUT /identity/v3/projects/eae17eee05c14fc19fe8538df9e7080f/users/c8c5987f72594a79adddec3c5eccc0ab/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:09.933524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e4629-4aa9-4301-bac0-f4856cab526f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:10.008174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a6a635daff452a99c82cb551c3cecc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:10.009847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a6a635daff452a99c82cb551c3cecc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:10.011429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a6a635daff452a99c82cb551c3cecc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:10.047589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e4629-4aa9-4301-bac0-f4856cab526f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:10.060405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bc3b5-3b62-4e3a-97a6-f7386bd00dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:10.061138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bc3b5-3b62-4e3a-97a6-f7386bd00dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.061456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bc3b5-3b62-4e3a-97a6-f7386bd00dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.061927 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bc3b5-3b62-4e3a-97a6-f7386bd00dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.062628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bc3b5-3b62-4e3a-97a6-f7386bd00dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:10.064332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bc3b5-3b62-4e3a-97a6-f7386bd00dbc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:10.084725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 886/1762] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:10.086198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bc3b5-3b62-4e3a-97a6-f7386bd00dbc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:10.088475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-428bc3b5-3b62-4e3a-97a6-f7386bd00dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:10.118178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bc3b5-3b62-4e3a-97a6-f7386bd00dbc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:10.127024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a6a635daff452a99c82cb551c3cecc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:10.127188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b1d2d-acda-474b-bd3b-0b3c305a03e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:10.127188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b1d2d-acda-474b-bd3b-0b3c305a03e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.127188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b1d2d-acda-474b-bd3b-0b3c305a03e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.127188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b1d2d-acda-474b-bd3b-0b3c305a03e3 admin admin] PATH_INFO: `/v3/projects/eae17eee05c14fc19fe8538df9e7080f/users/c8c5987f72594a79adddec3c5eccc0ab/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.127553 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b1d2d-acda-474b-bd3b-0b3c305a03e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae17eee05c14fc19fe8538df9e7080f, user_id=c8c5987f72594a79adddec3c5eccc0ab, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:10.129606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b1d2d-acda-474b-bd3b-0b3c305a03e3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:10.145700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 887/1763] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:10 2022] PUT /identity/v3/projects/eae17eee05c14fc19fe8538df9e7080f/users/c8c5987f72594a79adddec3c5eccc0ab/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:10.145931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cef09ed4-b044-4c8d-8321-5ebc825d3a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a6a635daff452a99c82cb551c3cecc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:10.163623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228cc6-7dd0-4928-89d8-bbc4bfc6220e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.164453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228cc6-7dd0-4928-89d8-bbc4bfc6220e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.164915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228cc6-7dd0-4928-89d8-bbc4bfc6220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.174833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 877/1764] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:09 2022] DELETE /identity/v3/users/12a6a635daff452a99c82cb551c3cecc => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:10.179575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69195df-a523-4456-a2c2-cd5e4bff1df1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.180171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69195df-a523-4456-a2c2-cd5e4bff1df1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.180784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69195df-a523-4456-a2c2-cd5e4bff1df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.208098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d69195df-a523-4456-a2c2-cd5e4bff1df1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:10.210258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d69195df-a523-4456-a2c2-cd5e4bff1df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:10.249917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5c228cc6-7dd0-4928-89d8-bbc4bfc6220e None None] MFA Rules not processed for user `c8c5987f72594a79adddec3c5eccc0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:10.251548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5c228cc6-7dd0-4928-89d8-bbc4bfc6220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:10.356736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 878/1765] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:10.369380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69195df-a523-4456-a2c2-cd5e4bff1df1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:10.371093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d69195df-a523-4456-a2c2-cd5e4bff1df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:10.406683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:10.450109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 888/1766] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:10.460785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228cc6-7dd0-4928-89d8-bbc4bfc6220e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:10.464166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5c228cc6-7dd0-4928-89d8-bbc4bfc6220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:10.468486 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69195df-a523-4456-a2c2-cd5e4bff1df1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:10.472657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f97ba4-76a8-4809-9ed1-7356ec2dc85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:10.473352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f97ba4-76a8-4809-9ed1-7356ec2dc85b service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.473649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f97ba4-76a8-4809-9ed1-7356ec2dc85b service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.473945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f97ba4-76a8-4809-9ed1-7356ec2dc85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.489698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f97ba4-76a8-4809-9ed1-7356ec2dc85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:10.491519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f97ba4-76a8-4809-9ed1-7356ec2dc85b service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:10.524179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 879/1767] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:10.559758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228cc6-7dd0-4928-89d8-bbc4bfc6220e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:10.564679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c565a8-1a4e-46b2-a9b1-ebb40fc49219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:10.565433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c565a8-1a4e-46b2-a9b1-ebb40fc49219 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.565813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c565a8-1a4e-46b2-a9b1-ebb40fc49219 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.566242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c565a8-1a4e-46b2-a9b1-ebb40fc49219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.578768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c565a8-1a4e-46b2-a9b1-ebb40fc49219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:10.580516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c565a8-1a4e-46b2-a9b1-ebb40fc49219 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:10.597557 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 889/1768] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:11.067131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f97ba4-76a8-4809-9ed1-7356ec2dc85b service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:11.069262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b5f97ba4-76a8-4809-9ed1-7356ec2dc85b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:11.096419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f97ba4-76a8-4809-9ed1-7356ec2dc85b service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:11.103355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:11.104167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:11.104508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:11.104955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] PATH_INFO: `/v3/projects/794f120571ed40e1bd6ddf4011bdd849` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:11.106483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=794f120571ed40e1bd6ddf4011bdd849)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:11.109020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:11.212672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794f120571ed40e1bd6ddf4011bdd849', 'request_id': 'req-870f9154-eded-412d-b7b5-ff9b51ca88ff'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.218192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794f120571ed40e1bd6ddf4011bdd849', 'request_id': 'req-870f9154-eded-412d-b7b5-ff9b51ca88ff'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.224313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794f120571ed40e1bd6ddf4011bdd849', 'request_id': 'req-870f9154-eded-412d-b7b5-ff9b51ca88ff'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.306556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] The token cache is being invalidate because project 794f120571ed40e1bd6ddf4011bdd849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:11.306849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.307547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.309063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 880/1769] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:11 2022] DELETE /identity/v3/projects/794f120571ed40e1bd6ddf4011bdd849 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:11.313937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c565a8-1a4e-46b2-a9b1-ebb40fc49219 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:11.325716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c565a8-1a4e-46b2-a9b1-ebb40fc49219 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:11.393829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:11.393829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:11.393829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:11.393829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] PATH_INFO: `/v3/users/65fbcb6af26b4cfd948cc606810b76c7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:11.395091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65fbcb6af26b4cfd948cc606810b76c7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:11.397397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:11.526613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:11.636585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65fbcb6af26b4cfd948cc606810b76c7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.637246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65fbcb6af26b4cfd948cc606810b76c7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.637852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65fbcb6af26b4cfd948cc606810b76c7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.646431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65fbcb6af26b4cfd948cc606810b76c7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.657504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9acb2c50-ee3a-4a29-95d0-c05958c65614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65fbcb6af26b4cfd948cc606810b76c7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:11.712038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 890/1770] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:11 2022] DELETE /identity/v3/users/65fbcb6af26b4cfd948cc606810b76c7 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:11.746378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:11.746378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:11.750670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f9154-eded-412d-b7b5-ff9b51ca88ff admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:11.827898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:11.828537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:11.828810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:11.829088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] PATH_INFO: `/v3/users/a5ab3b8e84fa429a91383f24214a88ce` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:11.830255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ab3b8e84fa429a91383f24214a88ce)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:11.832172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:12.071209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ab3b8e84fa429a91383f24214a88ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.071209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ab3b8e84fa429a91383f24214a88ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.071209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ab3b8e84fa429a91383f24214a88ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.209295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ab3b8e84fa429a91383f24214a88ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.218030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ab3b8e84fa429a91383f24214a88ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.235476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 881/1771] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:11 2022] DELETE /identity/v3/users/a5ab3b8e84fa429a91383f24214a88ce => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:12.240496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bf441-37b1-40f4-87dc-8da6ff5943d4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:12.241176 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bf441-37b1-40f4-87dc-8da6ff5943d4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:12.241635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bf441-37b1-40f4-87dc-8da6ff5943d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:12.245818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:12.247255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:12.263347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4d5bf441-37b1-40f4-87dc-8da6ff5943d4 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:12.266210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4d5bf441-37b1-40f4-87dc-8da6ff5943d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:12.330998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd0104-e55f-4dfc-952d-01416112944d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:12.336381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:12.337690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:12.338185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:12.338621 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] PATH_INFO: `/v3/projects/d15f287e716e4b1f89bf3dcdb9fdaf1a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:12.339965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d15f287e716e4b1f89bf3dcdb9fdaf1a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:12.342767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:12.346431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 891/1772] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:12.358903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bf441-37b1-40f4-87dc-8da6ff5943d4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:12.360567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4d5bf441-37b1-40f4-87dc-8da6ff5943d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:12.422691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bf441-37b1-40f4-87dc-8da6ff5943d4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:12.426547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2c9d7-46da-4245-ad3d-e76fa7b23780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:12.427213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c9d7-46da-4245-ad3d-e76fa7b23780 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:12.427503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c9d7-46da-4245-ad3d-e76fa7b23780 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:12.427778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c9d7-46da-4245-ad3d-e76fa7b23780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:12.438386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2c9d7-46da-4245-ad3d-e76fa7b23780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:12.439881 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2c9d7-46da-4245-ad3d-e76fa7b23780 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:12.450858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 892/1773] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:12.455876 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd15f287e716e4b1f89bf3dcdb9fdaf1a', 'request_id': 'req-ffad8983-0f95-421f-b8cf-114b27869b8c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.462447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd15f287e716e4b1f89bf3dcdb9fdaf1a', 'request_id': 'req-ffad8983-0f95-421f-b8cf-114b27869b8c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.469990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd15f287e716e4b1f89bf3dcdb9fdaf1a', 'request_id': 'req-ffad8983-0f95-421f-b8cf-114b27869b8c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.496086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:12.590185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] The token cache is being invalidate because project d15f287e716e4b1f89bf3dcdb9fdaf1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:12.590510 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.591160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ffad8983-0f95-421f-b8cf-114b27869b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:12.594985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 882/1774] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:12 2022] DELETE /identity/v3/projects/d15f287e716e4b1f89bf3dcdb9fdaf1a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:12.599522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d266713-a44c-4cf4-8cb5-896cf5a7d210 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:12.601056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d266713-a44c-4cf4-8cb5-896cf5a7d210 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:12.601595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d266713-a44c-4cf4-8cb5-896cf5a7d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:12.629799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7d266713-a44c-4cf4-8cb5-896cf5a7d210 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:12.631091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7d266713-a44c-4cf4-8cb5-896cf5a7d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:12.712897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 893/1775] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:12.720173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ef459-d42b-4b6f-bcec-1abe2530ab75 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:12.720518 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ef459-d42b-4b6f-bcec-1abe2530ab75 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:12.720856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ef459-d42b-4b6f-bcec-1abe2530ab75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:12.744966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cf2ef459-d42b-4b6f-bcec-1abe2530ab75 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:12.746331 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cf2ef459-d42b-4b6f-bcec-1abe2530ab75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:12.759070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 883/1776] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:12.761264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d266713-a44c-4cf4-8cb5-896cf5a7d210 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:12.772118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d266713-a44c-4cf4-8cb5-896cf5a7d210 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:12.777616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045fc8-0dba-4e3f-878e-286d2f59d308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:12.778284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045fc8-0dba-4e3f-878e-286d2f59d308 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:12.778566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045fc8-0dba-4e3f-878e-286d2f59d308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:12.778840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045fc8-0dba-4e3f-878e-286d2f59d308 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:12.779598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045fc8-0dba-4e3f-878e-286d2f59d308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:25:12.779988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045fc8-0dba-4e3f-878e-286d2f59d308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:12.781657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045fc8-0dba-4e3f-878e-286d2f59d308 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:12.799172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 894/1777] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:25:12.813859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ef459-d42b-4b6f-bcec-1abe2530ab75 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:12.822002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ef459-d42b-4b6f-bcec-1abe2530ab75 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:12.827756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525b98e-9c22-433e-a1ae-7d561024974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:12.831726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525b98e-9c22-433e-a1ae-7d561024974e admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:12.831726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525b98e-9c22-433e-a1ae-7d561024974e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:12.831726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525b98e-9c22-433e-a1ae-7d561024974e admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:12.831726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525b98e-9c22-433e-a1ae-7d561024974e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:12.831726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525b98e-9c22-433e-a1ae-7d561024974e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:12.959152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 884/1778] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:12 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:12.960248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045fc8-0dba-4e3f-878e-286d2f59d308 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:12.965959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525b98e-9c22-433e-a1ae-7d561024974e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:12.967939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0525b98e-9c22-433e-a1ae-7d561024974e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:12.976887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045fc8-0dba-4e3f-878e-286d2f59d308 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:13.057558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525b98e-9c22-433e-a1ae-7d561024974e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:13.064195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:13.068077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.068297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7964d9a-3869-4f6c-a41c-e1d7e394a8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:13.068435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.068435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] PATH_INFO: `/v3/projects/7e11f56f3d1d4ed1874d78e5e215e213` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.068584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7964d9a-3869-4f6c-a41c-e1d7e394a8fb admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.068584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7964d9a-3869-4f6c-a41c-e1d7e394a8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.068584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7964d9a-3869-4f6c-a41c-e1d7e394a8fb admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.068792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e11f56f3d1d4ed1874d78e5e215e213)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:13.068871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7964d9a-3869-4f6c-a41c-e1d7e394a8fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:13.070273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7964d9a-3869-4f6c-a41c-e1d7e394a8fb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:13.070384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:13.197423 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 895/1779] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:12 2022] POST /identity/v3/users => generated 282 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:13.199020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7964d9a-3869-4f6c-a41c-e1d7e394a8fb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:13.206200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e11f56f3d1d4ed1874d78e5e215e213', 'request_id': 'req-0cf875b0-2982-4577-86a6-b0ebcf79e741'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.209171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7964d9a-3869-4f6c-a41c-e1d7e394a8fb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:13.211879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e11f56f3d1d4ed1874d78e5e215e213', 'request_id': 'req-0cf875b0-2982-4577-86a6-b0ebcf79e741'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.218659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e11f56f3d1d4ed1874d78e5e215e213', 'request_id': 'req-0cf875b0-2982-4577-86a6-b0ebcf79e741'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.264251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c891794-d494-48be-ba75-1c8749f2fd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:13.264863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c891794-d494-48be-ba75-1c8749f2fd56 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.265140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c891794-d494-48be-ba75-1c8749f2fd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.265409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c891794-d494-48be-ba75-1c8749f2fd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.266096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c891794-d494-48be-ba75-1c8749f2fd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:13.267787 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c891794-d494-48be-ba75-1c8749f2fd56 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:13.277638 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 896/1780] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:13.282235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c891794-d494-48be-ba75-1c8749f2fd56 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:13.292649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c891794-d494-48be-ba75-1c8749f2fd56 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:13.297257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] The token cache is being invalidate because project 7e11f56f3d1d4ed1874d78e5e215e213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:13.297550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.298040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4126f9-059d-4c4e-ac80-7a47fde62773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:13.298198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.298651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4126f9-059d-4c4e-ac80-7a47fde62773 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.298934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4126f9-059d-4c4e-ac80-7a47fde62773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.299297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4126f9-059d-4c4e-ac80-7a47fde62773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.300110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4126f9-059d-4c4e-ac80-7a47fde62773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:13.300224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 885/1781] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:12 2022] DELETE /identity/v3/projects/7e11f56f3d1d4ed1874d78e5e215e213 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:13.301513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4126f9-059d-4c4e-ac80-7a47fde62773 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:13.303775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:13.305291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:13.315436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 897/1782] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:13.316501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4126f9-059d-4c4e-ac80-7a47fde62773 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:13.318233 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8b4126f9-059d-4c4e-ac80-7a47fde62773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:13.319296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf875b0-2982-4577-86a6-b0ebcf79e741 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:13.324122 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:13.324892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.325184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.325461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] PATH_INFO: `/v3/users/c8c5987f72594a79adddec3c5eccc0ab` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.326698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c5987f72594a79adddec3c5eccc0ab)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:13.331257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4126f9-059d-4c4e-ac80-7a47fde62773 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:13.331383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:13.343926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6f46c-8dda-469c-8ec3-382bb1bacee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:13.344616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6f46c-8dda-469c-8ec3-382bb1bacee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.344895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6f46c-8dda-469c-8ec3-382bb1bacee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.345166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6f46c-8dda-469c-8ec3-382bb1bacee2 admin admin] PATH_INFO: `/v3/projects/091d52f2209a4a79a1f5f8f27db8ebb5/users/c3f6348ab76c440abea1a1bf0a5b276f/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.362397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6f46c-8dda-469c-8ec3-382bb1bacee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=091d52f2209a4a79a1f5f8f27db8ebb5, user_id=c3f6348ab76c440abea1a1bf0a5b276f, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:13.368322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6f46c-8dda-469c-8ec3-382bb1bacee2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:13.455514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:13.462120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 898/1783] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:13 2022] PUT /identity/v3/projects/091d52f2209a4a79a1f5f8f27db8ebb5/users/c3f6348ab76c440abea1a1bf0a5b276f/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:13.475902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbb0f8-6372-4e8e-866a-6ecc3d64eaec None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.476195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbb0f8-6372-4e8e-866a-6ecc3d64eaec None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.476466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbb0f8-6372-4e8e-866a-6ecc3d64eaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.507179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f8fbb0f8-6372-4e8e-866a-6ecc3d64eaec None None] MFA Rules not processed for user `c3f6348ab76c440abea1a1bf0a5b276f`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:13.508532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f8fbb0f8-6372-4e8e-866a-6ecc3d64eaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:13.563663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c5987f72594a79adddec3c5eccc0ab'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.564518 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c5987f72594a79adddec3c5eccc0ab'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.565226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c5987f72594a79adddec3c5eccc0ab'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.579796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 899/1784] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:13.593232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbb0f8-6372-4e8e-866a-6ecc3d64eaec None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:13.594703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f8fbb0f8-6372-4e8e-866a-6ecc3d64eaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:13.628213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c5987f72594a79adddec3c5eccc0ab'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.636023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c5987f72594a79adddec3c5eccc0ab'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:13.653900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 886/1785] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:13 2022] DELETE /identity/v3/users/c8c5987f72594a79adddec3c5eccc0ab => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:13.663080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbb0f8-6372-4e8e-866a-6ecc3d64eaec None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:13.723115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a82f9-f5e4-4985-9ec9-df33aa79c167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:13.723737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a82f9-f5e4-4985-9ec9-df33aa79c167 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.724014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a82f9-f5e4-4985-9ec9-df33aa79c167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.724287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a82f9-f5e4-4985-9ec9-df33aa79c167 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.724919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a82f9-f5e4-4985-9ec9-df33aa79c167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:13.726880 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a82f9-f5e4-4985-9ec9-df33aa79c167 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:13.846781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 900/1786] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:13 2022] POST /identity/v3/projects => generated 369 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:13.850683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:13.867839 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60a5fc-1df8-447c-a725-8b0d8f6e7dcd admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:13.937077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89bff7-128e-4c9d-8890-fc7259bd6256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:13.937675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89bff7-128e-4c9d-8890-fc7259bd6256 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.938691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89bff7-128e-4c9d-8890-fc7259bd6256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.939099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89bff7-128e-4c9d-8890-fc7259bd6256 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.939859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89bff7-128e-4c9d-8890-fc7259bd6256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:13.942738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89bff7-128e-4c9d-8890-fc7259bd6256 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:14.006890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 887/1787] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:13 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:14.010816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a82f9-f5e4-4985-9ec9-df33aa79c167 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:14.023481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a82f9-f5e4-4985-9ec9-df33aa79c167 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:14.036068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c15146-1985-4b96-ab95-98a70bab69ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:14.036958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15146-1985-4b96-ab95-98a70bab69ae admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.037335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15146-1985-4b96-ab95-98a70bab69ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.037819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15146-1985-4b96-ab95-98a70bab69ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.038867 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c15146-1985-4b96-ab95-98a70bab69ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:14.040930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c15146-1985-4b96-ab95-98a70bab69ae admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:14.062680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 901/1788] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:14.066641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89bff7-128e-4c9d-8890-fc7259bd6256 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:14.083035 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89bff7-128e-4c9d-8890-fc7259bd6256 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:14.106598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b337c5-9a45-451b-b162-30333c57a768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:14.109234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b337c5-9a45-451b-b162-30333c57a768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.115238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b337c5-9a45-451b-b162-30333c57a768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.115238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b337c5-9a45-451b-b162-30333c57a768 admin admin] PATH_INFO: `/v3/projects/66f49554fcc54288a9a4ca4d1b38e5ba/users/290d77d1a52b4d45a314987bcad1efa2/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.127042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b337c5-9a45-451b-b162-30333c57a768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f49554fcc54288a9a4ca4d1b38e5ba, user_id=290d77d1a52b4d45a314987bcad1efa2, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:14.142039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b337c5-9a45-451b-b162-30333c57a768 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:14.223999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 888/1789] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:14 2022] PUT /identity/v3/projects/66f49554fcc54288a9a4ca4d1b38e5ba/users/290d77d1a52b4d45a314987bcad1efa2/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:14.228096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c15146-1985-4b96-ab95-98a70bab69ae admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:14.239621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c15146-1985-4b96-ab95-98a70bab69ae admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:14.278070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b337c5-9a45-451b-b162-30333c57a768 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:14.282765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-33b337c5-9a45-451b-b162-30333c57a768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:14.363655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51fed8-c909-4697-a76a-fee538a4cedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:14.364299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51fed8-c909-4697-a76a-fee538a4cedb admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.364571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51fed8-c909-4697-a76a-fee538a4cedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.364871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51fed8-c909-4697-a76a-fee538a4cedb admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.365530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51fed8-c909-4697-a76a-fee538a4cedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:14.367203 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51fed8-c909-4697-a76a-fee538a4cedb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:14.393926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 902/1790] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:14.396278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51fed8-c909-4697-a76a-fee538a4cedb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:14.412257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51fed8-c909-4697-a76a-fee538a4cedb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:14.420802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef75862-bc98-4867-a788-e65bb9824e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:14.422379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef75862-bc98-4867-a788-e65bb9824e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.423243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef75862-bc98-4867-a788-e65bb9824e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.423613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef75862-bc98-4867-a788-e65bb9824e1e admin admin] PATH_INFO: `/v3/projects/66f49554fcc54288a9a4ca4d1b38e5ba/users/290d77d1a52b4d45a314987bcad1efa2/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.425407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b337c5-9a45-451b-b162-30333c57a768 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:14.425771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef75862-bc98-4867-a788-e65bb9824e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f49554fcc54288a9a4ca4d1b38e5ba, user_id=290d77d1a52b4d45a314987bcad1efa2, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:14.429938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef75862-bc98-4867-a788-e65bb9824e1e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:14.431561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:14.432239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.432524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.432802 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] PATH_INFO: `/v3/projects/eae17eee05c14fc19fe8538df9e7080f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.434309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae17eee05c14fc19fe8538df9e7080f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:14.438378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:14.462784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 903/1791] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:14 2022] PUT /identity/v3/projects/66f49554fcc54288a9a4ca4d1b38e5ba/users/290d77d1a52b4d45a314987bcad1efa2/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:14.466843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:14.474239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613abf5-f4b8-4c44-ae03-c5036eb41b69 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.474595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613abf5-f4b8-4c44-ae03-c5036eb41b69 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.474872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613abf5-f4b8-4c44-ae03-c5036eb41b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.530600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b613abf5-f4b8-4c44-ae03-c5036eb41b69 None None] MFA Rules not processed for user `290d77d1a52b4d45a314987bcad1efa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:14.531860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b613abf5-f4b8-4c44-ae03-c5036eb41b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:14.543122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:14.595700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae17eee05c14fc19fe8538df9e7080f', 'request_id': 'req-99299dc8-0cb8-4f8b-8a7d-47631660fa41'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:14.638648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae17eee05c14fc19fe8538df9e7080f', 'request_id': 'req-99299dc8-0cb8-4f8b-8a7d-47631660fa41'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:14.646233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae17eee05c14fc19fe8538df9e7080f', 'request_id': 'req-99299dc8-0cb8-4f8b-8a7d-47631660fa41'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:14.651586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 904/1792] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:14.666366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613abf5-f4b8-4c44-ae03-c5036eb41b69 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:14.668194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b613abf5-f4b8-4c44-ae03-c5036eb41b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:14.746553 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613abf5-f4b8-4c44-ae03-c5036eb41b69 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:14.768391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] The token cache is being invalidate because project eae17eee05c14fc19fe8538df9e7080f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:14.768757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:14.769491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-99299dc8-0cb8-4f8b-8a7d-47631660fa41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:14.774460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 889/1793] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:14 2022] DELETE /identity/v3/projects/eae17eee05c14fc19fe8538df9e7080f => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:14.776470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fbc5c-cac7-473f-869c-e601f13efabf None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.776760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fbc5c-cac7-473f-869c-e601f13efabf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.777038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fbc5c-cac7-473f-869c-e601f13efabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.802694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-708fbc5c-cac7-473f-869c-e601f13efabf None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:14.803982 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-708fbc5c-cac7-473f-869c-e601f13efabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:14.815115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab6149-f57a-43d9-a108-3d22dd498a5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:14.817322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab6149-f57a-43d9-a108-3d22dd498a5d service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.817322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab6149-f57a-43d9-a108-3d22dd498a5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.817322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab6149-f57a-43d9-a108-3d22dd498a5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.818576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d7ab6149-f57a-43d9-a108-3d22dd498a5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:14.879351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 890/1794] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:14.887204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef630bd-7520-4777-b345-12646c344533 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.891971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef630bd-7520-4777-b345-12646c344533 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.891971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef630bd-7520-4777-b345-12646c344533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.920134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8ef630bd-7520-4777-b345-12646c344533 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:14.921435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8ef630bd-7520-4777-b345-12646c344533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:14.925810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab6149-f57a-43d9-a108-3d22dd498a5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:14.932002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab6149-f57a-43d9-a108-3d22dd498a5d service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:14.932274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 891/1795] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:14.936746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef630bd-7520-4777-b345-12646c344533 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:14.941458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 905/1796] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:14.949024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef630bd-7520-4777-b345-12646c344533 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:14.954620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f5f26-7bf9-47ec-9ec1-25fdd97bf9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:14.955295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f5f26-7bf9-47ec-9ec1-25fdd97bf9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.955577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f5f26-7bf9-47ec-9ec1-25fdd97bf9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.955853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f5f26-7bf9-47ec-9ec1-25fdd97bf9f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.956505 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f5f26-7bf9-47ec-9ec1-25fdd97bf9f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:25:14.956813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f5f26-7bf9-47ec-9ec1-25fdd97bf9f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:14.958505 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f5f26-7bf9-47ec-9ec1-25fdd97bf9f8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:14.975853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 892/1797] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:25:14.978873 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab6149-f57a-43d9-a108-3d22dd498a5d service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:14.993293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab6149-f57a-43d9-a108-3d22dd498a5d service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:15.008319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0013156-e238-402d-80cb-7f916f42a43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:15.008889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0013156-e238-402d-80cb-7f916f42a43d admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:15.009174 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0013156-e238-402d-80cb-7f916f42a43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:15.009786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0013156-e238-402d-80cb-7f916f42a43d admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:15.010325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0013156-e238-402d-80cb-7f916f42a43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:15.012578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0013156-e238-402d-80cb-7f916f42a43d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:15.290994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 906/1798] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:14 2022] POST /identity/v3/projects => generated 365 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:15.312703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f5f26-7bf9-47ec-9ec1-25fdd97bf9f8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:15.329841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f5f26-7bf9-47ec-9ec1-25fdd97bf9f8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:15.401581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224ba27-b621-45c1-ad38-d08127557327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:15.402507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224ba27-b621-45c1-ad38-d08127557327 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:15.402807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224ba27-b621-45c1-ad38-d08127557327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:15.403082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224ba27-b621-45c1-ad38-d08127557327 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:15.403713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224ba27-b621-45c1-ad38-d08127557327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:15.406243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224ba27-b621-45c1-ad38-d08127557327 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:16.254996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 893/1799] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:15 2022] POST /identity/v3/users => generated 283 bytes in 940 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:16.256374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0013156-e238-402d-80cb-7f916f42a43d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:16.307591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0013156-e238-402d-80cb-7f916f42a43d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:16.323466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd000354-9957-4409-a321-602e2be809d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:16.323662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd000354-9957-4409-a321-602e2be809d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:16.331132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd000354-9957-4409-a321-602e2be809d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:16.338205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd000354-9957-4409-a321-602e2be809d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:16.338205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd000354-9957-4409-a321-602e2be809d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:16.338205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd000354-9957-4409-a321-602e2be809d6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:16.359086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224ba27-b621-45c1-ad38-d08127557327 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:16.363703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c224ba27-b621-45c1-ad38-d08127557327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:16.379296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224ba27-b621-45c1-ad38-d08127557327 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:16.383014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 907/1800] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:16.384791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd000354-9957-4409-a321-602e2be809d6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:16.393324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:16.394410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:16.394877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:16.396623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] PATH_INFO: `/v3/users/7cf99bf2d43b4b0a8d2cb48aead512bf` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:16.399149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf99bf2d43b4b0a8d2cb48aead512bf)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:16.405055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:16.422539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd000354-9957-4409-a321-602e2be809d6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:16.446379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb7b1-a318-4bef-9934-96f3f5628bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:16.447434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb7b1-a318-4bef-9934-96f3f5628bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:16.447930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb7b1-a318-4bef-9934-96f3f5628bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:16.448376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb7b1-a318-4bef-9934-96f3f5628bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:16.449280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb7b1-a318-4bef-9934-96f3f5628bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:16.451382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb7b1-a318-4bef-9934-96f3f5628bc0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:16.472392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 908/1801] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:16.476799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb7b1-a318-4bef-9934-96f3f5628bc0 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:16.508189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb7b1-a318-4bef-9934-96f3f5628bc0 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:16.525492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341de909-2a25-4ac7-9975-ae5dcef79e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:16.526550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341de909-2a25-4ac7-9975-ae5dcef79e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:16.526975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341de909-2a25-4ac7-9975-ae5dcef79e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:16.527387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341de909-2a25-4ac7-9975-ae5dcef79e11 admin admin] PATH_INFO: `/v3/projects/9a205b5137a1474c82f08233653fa8d0/users/0121ccb5874a447bbc2eef3beb139346/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:16.599928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341de909-2a25-4ac7-9975-ae5dcef79e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a205b5137a1474c82f08233653fa8d0, user_id=0121ccb5874a447bbc2eef3beb139346, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:16.602874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341de909-2a25-4ac7-9975-ae5dcef79e11 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:16.713857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 909/1802] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:16 2022] PUT /identity/v3/projects/9a205b5137a1474c82f08233653fa8d0/users/0121ccb5874a447bbc2eef3beb139346/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:16.720245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5548649-9815-4e10-8545-d1527525c063 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:16.720534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5548649-9815-4e10-8545-d1527525c063 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:16.720807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5548649-9815-4e10-8545-d1527525c063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:16.731933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf99bf2d43b4b0a8d2cb48aead512bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:16.737609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf99bf2d43b4b0a8d2cb48aead512bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:16.741315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf99bf2d43b4b0a8d2cb48aead512bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:16.787882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b5548649-9815-4e10-8545-d1527525c063 None None] MFA Rules not processed for user `0121ccb5874a447bbc2eef3beb139346`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:16.789132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b5548649-9815-4e10-8545-d1527525c063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:16.814668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf99bf2d43b4b0a8d2cb48aead512bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:16.829288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64216f27-1042-4b94-a3d5-808e85bca0e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf99bf2d43b4b0a8d2cb48aead512bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:16.859126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 894/1803] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:16 2022] DELETE /identity/v3/users/7cf99bf2d43b4b0a8d2cb48aead512bf => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:16.860745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562e0de-dd4f-4cd8-a4bd-07d0cbc0bc41 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:16.861088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562e0de-dd4f-4cd8-a4bd-07d0cbc0bc41 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:16.861442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562e0de-dd4f-4cd8-a4bd-07d0cbc0bc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:16.902807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9562e0de-dd4f-4cd8-a4bd-07d0cbc0bc41 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:16.905515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9562e0de-dd4f-4cd8-a4bd-07d0cbc0bc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:16.931011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 910/1804] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:16.944100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5548649-9815-4e10-8545-d1527525c063 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:16.945681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b5548649-9815-4e10-8545-d1527525c063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:17.014144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 895/1805] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:17.023567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562e0de-dd4f-4cd8-a4bd-07d0cbc0bc41 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:17.025556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9562e0de-dd4f-4cd8-a4bd-07d0cbc0bc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:17.055255 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5548649-9815-4e10-8545-d1527525c063 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:17.062201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dde0d-6ab2-47a9-8822-2e682687d7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:17.063042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dde0d-6ab2-47a9-8822-2e682687d7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:17.063641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dde0d-6ab2-47a9-8822-2e682687d7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:17.064082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dde0d-6ab2-47a9-8822-2e682687d7d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:17.064943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0dde0d-6ab2-47a9-8822-2e682687d7d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:17.067624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0dde0d-6ab2-47a9-8822-2e682687d7d9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:17.116278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562e0de-dd4f-4cd8-a4bd-07d0cbc0bc41 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:17.129214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f50d0-75bc-4759-8fdd-4e6fd5515868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:17.129850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f50d0-75bc-4759-8fdd-4e6fd5515868 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:17.130456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f50d0-75bc-4759-8fdd-4e6fd5515868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:17.130456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f50d0-75bc-4759-8fdd-4e6fd5515868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:17.143288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f50d0-75bc-4759-8fdd-4e6fd5515868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:17.145464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f50d0-75bc-4759-8fdd-4e6fd5515868 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:17.161678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 896/1806] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:17.201125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 911/1807] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:16 2022] POST /identity/v3/projects => generated 363 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:17.203539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f50d0-75bc-4759-8fdd-4e6fd5515868 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:17.220782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f50d0-75bc-4759-8fdd-4e6fd5515868 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:17.221995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:17.304211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:17.358817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4f239-899a-480e-b494-fada4271dfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:17.359590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4f239-899a-480e-b494-fada4271dfe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:17.360207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4f239-899a-480e-b494-fada4271dfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:17.360207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4f239-899a-480e-b494-fada4271dfe0 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:17.361053 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4f239-899a-480e-b494-fada4271dfe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:17.363204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4f239-899a-480e-b494-fada4271dfe0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:17.483354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 897/1808] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:17 2022] POST /identity/v3/users => generated 288 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:17.484616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dde0d-6ab2-47a9-8822-2e682687d7d9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:17.497654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dde0d-6ab2-47a9-8822-2e682687d7d9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:17.511265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94855f-2a14-49f5-8fe5-e987af3eb9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:17.512009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94855f-2a14-49f5-8fe5-e987af3eb9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:17.512393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94855f-2a14-49f5-8fe5-e987af3eb9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:17.526399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94855f-2a14-49f5-8fe5-e987af3eb9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:17.526399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94855f-2a14-49f5-8fe5-e987af3eb9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:17.526399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94855f-2a14-49f5-8fe5-e987af3eb9cb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:17.558617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 912/1809] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:17.564190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4f239-899a-480e-b494-fada4271dfe0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:17.577117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4f239-899a-480e-b494-fada4271dfe0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:17.593739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9777d9a-3832-42ad-adaa-cfdff8c7c748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:17.595141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9777d9a-3832-42ad-adaa-cfdff8c7c748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:17.595814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9777d9a-3832-42ad-adaa-cfdff8c7c748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:17.595814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9777d9a-3832-42ad-adaa-cfdff8c7c748 admin admin] PATH_INFO: `/v3/projects/64ddde3cf4bf419ba4634151d1b9a7bb/users/1d77b799691d43f697f0b4cd2b032004/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:17.645710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9777d9a-3832-42ad-adaa-cfdff8c7c748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ddde3cf4bf419ba4634151d1b9a7bb, user_id=1d77b799691d43f697f0b4cd2b032004, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:17.662786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9777d9a-3832-42ad-adaa-cfdff8c7c748 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:17.776358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 898/1810] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:17 2022] PUT /identity/v3/projects/64ddde3cf4bf419ba4634151d1b9a7bb/users/1d77b799691d43f697f0b4cd2b032004/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:17.780896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94855f-2a14-49f5-8fe5-e987af3eb9cb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:17.815350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94855f-2a14-49f5-8fe5-e987af3eb9cb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:17.954904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc0f39-d368-4212-a1c3-20e077d05451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:17.955619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc0f39-d368-4212-a1c3-20e077d05451 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:17.956027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc0f39-d368-4212-a1c3-20e077d05451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:17.956308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc0f39-d368-4212-a1c3-20e077d05451 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:17.957114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc0f39-d368-4212-a1c3-20e077d05451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:17.959191 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc0f39-d368-4212-a1c3-20e077d05451 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:17.977788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 913/1811] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:17.980390 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9777d9a-3832-42ad-adaa-cfdff8c7c748 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:18.002352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9777d9a-3832-42ad-adaa-cfdff8c7c748 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:18.015557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc02ff-4420-4c29-b850-e606faf20034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:18.016522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc02ff-4420-4c29-b850-e606faf20034 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:18.021313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc02ff-4420-4c29-b850-e606faf20034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:18.021933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc02ff-4420-4c29-b850-e606faf20034 admin admin] PATH_INFO: `/v3/projects/64ddde3cf4bf419ba4634151d1b9a7bb/users/1d77b799691d43f697f0b4cd2b032004/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:18.024652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc02ff-4420-4c29-b850-e606faf20034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ddde3cf4bf419ba4634151d1b9a7bb, user_id=1d77b799691d43f697f0b4cd2b032004, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:18.028393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc02ff-4420-4c29-b850-e606faf20034 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:18.056795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 899/1812] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:17 2022] PUT /identity/v3/projects/64ddde3cf4bf419ba4634151d1b9a7bb/users/1d77b799691d43f697f0b4cd2b032004/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:18.078654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45859f-1cb0-45b1-8ba1-0c1046656430 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:18.078654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45859f-1cb0-45b1-8ba1-0c1046656430 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:18.078654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45859f-1cb0-45b1-8ba1-0c1046656430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:18.178961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3d45859f-1cb0-45b1-8ba1-0c1046656430 None None] MFA Rules not processed for user `1d77b799691d43f697f0b4cd2b032004`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:18.183491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3d45859f-1cb0-45b1-8ba1-0c1046656430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:18.190170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:18.349773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 914/1813] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:18.431563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc02ff-4420-4c29-b850-e606faf20034 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:18.434005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-12bc02ff-4420-4c29-b850-e606faf20034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:18.494496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:18.542113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc02ff-4420-4c29-b850-e606faf20034 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:18.549324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3528c-549f-40f9-87fb-e76436702ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:18.550097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3528c-549f-40f9-87fb-e76436702ecd service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:18.550477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3528c-549f-40f9-87fb-e76436702ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:18.551030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3528c-549f-40f9-87fb-e76436702ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:18.561825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3528c-549f-40f9-87fb-e76436702ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:18.565589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3528c-549f-40f9-87fb-e76436702ecd service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:18.573742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 900/1814] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:18.642112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45859f-1cb0-45b1-8ba1-0c1046656430 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:18.643608 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3d45859f-1cb0-45b1-8ba1-0c1046656430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:18.720463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45859f-1cb0-45b1-8ba1-0c1046656430 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:18.725053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:18.725688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:18.725998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:18.726281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] PATH_INFO: `/v3/projects/6cb495e455b848aba1d66a45c85f1f61` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:18.727431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cb495e455b848aba1d66a45c85f1f61)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:18.729350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:18.886726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cb495e455b848aba1d66a45c85f1f61', 'request_id': 'req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:18.892600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cb495e455b848aba1d66a45c85f1f61', 'request_id': 'req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:18.954988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3528c-549f-40f9-87fb-e76436702ecd service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:18.956440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2be3528c-549f-40f9-87fb-e76436702ecd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:18.967454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3528c-549f-40f9-87fb-e76436702ecd service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:19.019705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cb495e455b848aba1d66a45c85f1f61', 'request_id': 'req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:19.023051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6bb29-3b63-40e5-bdd5-1578b8d6fe32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:19.023763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6bb29-3b63-40e5-bdd5-1578b8d6fe32 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.024099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6bb29-3b63-40e5-bdd5-1578b8d6fe32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.024440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6bb29-3b63-40e5-bdd5-1578b8d6fe32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.032687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] The token cache is being invalidate because project 6cb495e455b848aba1d66a45c85f1f61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:19.032975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:19.035407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0bb32ae6-cc22-47cb-9b5f-99f6a5319141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:19.037794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 915/1815] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:18 2022] DELETE /identity/v3/projects/6cb495e455b848aba1d66a45c85f1f61 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:19.044423 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed85e8-ac91-44d5-a0cd-b0443404ce72 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.044638 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed85e8-ac91-44d5-a0cd-b0443404ce72 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.044920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed85e8-ac91-44d5-a0cd-b0443404ce72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.069405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d7ed85e8-ac91-44d5-a0cd-b0443404ce72 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:19.071132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d7ed85e8-ac91-44d5-a0cd-b0443404ce72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:19.096112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6bb29-3b63-40e5-bdd5-1578b8d6fe32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:19.097816 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6bb29-3b63-40e5-bdd5-1578b8d6fe32 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:19.108550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 901/1816] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:19.140007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:19.145725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 916/1817] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:19.153999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26fbf4-0982-4fcc-adb0-a29ac99f3c12 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.154920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26fbf4-0982-4fcc-adb0-a29ac99f3c12 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.155975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26fbf4-0982-4fcc-adb0-a29ac99f3c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.184637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3e26fbf4-0982-4fcc-adb0-a29ac99f3c12 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:19.186404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3e26fbf4-0982-4fcc-adb0-a29ac99f3c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:19.206356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 902/1818] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:19.210367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed85e8-ac91-44d5-a0cd-b0443404ce72 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:19.221801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed85e8-ac91-44d5-a0cd-b0443404ce72 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:19.226794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759d1b4-aa4b-4bab-aa42-d03b74b74e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:19.227402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759d1b4-aa4b-4bab-aa42-d03b74b74e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.227676 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759d1b4-aa4b-4bab-aa42-d03b74b74e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.227952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759d1b4-aa4b-4bab-aa42-d03b74b74e0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.230040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759d1b4-aa4b-4bab-aa42-d03b74b74e0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:25:19.230040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759d1b4-aa4b-4bab-aa42-d03b74b74e0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:19.230574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759d1b4-aa4b-4bab-aa42-d03b74b74e0e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:19.244538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 917/1819] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:25:19.249683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26fbf4-0982-4fcc-adb0-a29ac99f3c12 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:19.259297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26fbf4-0982-4fcc-adb0-a29ac99f3c12 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:19.264999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcf837-8353-4e7a-bb44-406bb167c506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:19.265752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcf837-8353-4e7a-bb44-406bb167c506 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.266067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcf837-8353-4e7a-bb44-406bb167c506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.266346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcf837-8353-4e7a-bb44-406bb167c506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.267086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcf837-8353-4e7a-bb44-406bb167c506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:19.269111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcf837-8353-4e7a-bb44-406bb167c506 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:19.420389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 903/1820] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:19 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:19.423385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759d1b4-aa4b-4bab-aa42-d03b74b74e0e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:19.434066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759d1b4-aa4b-4bab-aa42-d03b74b74e0e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:19.510428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11255aea-3093-4e6c-b5f6-f28060ac9bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:19.513065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11255aea-3093-4e6c-b5f6-f28060ac9bac admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.513351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11255aea-3093-4e6c-b5f6-f28060ac9bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.513627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11255aea-3093-4e6c-b5f6-f28060ac9bac admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.515350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11255aea-3093-4e6c-b5f6-f28060ac9bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:19.518226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11255aea-3093-4e6c-b5f6-f28060ac9bac admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:19.684751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 918/1821] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:19 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:19.685990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcf837-8353-4e7a-bb44-406bb167c506 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:19.706339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcf837-8353-4e7a-bb44-406bb167c506 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:19.714933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f310b05-27f4-42ef-b77e-576e5ef08b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:19.716117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f310b05-27f4-42ef-b77e-576e5ef08b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.716527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f310b05-27f4-42ef-b77e-576e5ef08b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.718191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f310b05-27f4-42ef-b77e-576e5ef08b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.719065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f310b05-27f4-42ef-b77e-576e5ef08b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:19.720993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f310b05-27f4-42ef-b77e-576e5ef08b03 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:19.745455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 904/1822] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:19.749650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11255aea-3093-4e6c-b5f6-f28060ac9bac admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:19.765764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11255aea-3093-4e6c-b5f6-f28060ac9bac admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:19.771352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983b71-33f8-48c6-bc12-ebf35bcd06d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:19.772257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983b71-33f8-48c6-bc12-ebf35bcd06d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.772686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983b71-33f8-48c6-bc12-ebf35bcd06d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.772969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983b71-33f8-48c6-bc12-ebf35bcd06d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.773649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12983b71-33f8-48c6-bc12-ebf35bcd06d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:19.775310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12983b71-33f8-48c6-bc12-ebf35bcd06d9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:19.788215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 919/1823] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:19.793316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f310b05-27f4-42ef-b77e-576e5ef08b03 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:19.805182 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f310b05-27f4-42ef-b77e-576e5ef08b03 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:19.810455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d318a06-b847-4c8f-b52e-2be8206f7594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:19.811193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d318a06-b847-4c8f-b52e-2be8206f7594 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.811476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d318a06-b847-4c8f-b52e-2be8206f7594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.811867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d318a06-b847-4c8f-b52e-2be8206f7594 admin admin] PATH_INFO: `/v3/projects/3e00b6aa41da4f77af9e35c2d4e5c1ce/users/21710b1a9c054c2bae5b48fc20d5d9b4/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.828286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d318a06-b847-4c8f-b52e-2be8206f7594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e00b6aa41da4f77af9e35c2d4e5c1ce, user_id=21710b1a9c054c2bae5b48fc20d5d9b4, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:19.831367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d318a06-b847-4c8f-b52e-2be8206f7594 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:19.927777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:19.934118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 905/1824] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:19 2022] PUT /identity/v3/projects/3e00b6aa41da4f77af9e35c2d4e5c1ce/users/21710b1a9c054c2bae5b48fc20d5d9b4/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:19.946567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0a80-4dd6-462a-a05a-fb5f56526969 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.946918 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0a80-4dd6-462a-a05a-fb5f56526969 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.947257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0a80-4dd6-462a-a05a-fb5f56526969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.975995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-59ae0a80-4dd6-462a-a05a-fb5f56526969 None None] MFA Rules not processed for user `21710b1a9c054c2bae5b48fc20d5d9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:19.977210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-59ae0a80-4dd6-462a-a05a-fb5f56526969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:20.062737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 920/1825] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:20.067455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d318a06-b847-4c8f-b52e-2be8206f7594 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:20.087936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:20.162136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d318a06-b847-4c8f-b52e-2be8206f7594 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:20.167714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec1cab-a432-4d9f-b364-50bd8a862828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:20.168349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1cab-a432-4d9f-b364-50bd8a862828 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:20.168727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1cab-a432-4d9f-b364-50bd8a862828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:20.168881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1cab-a432-4d9f-b364-50bd8a862828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:20.169787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec1cab-a432-4d9f-b364-50bd8a862828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:20.171778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec1cab-a432-4d9f-b364-50bd8a862828 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:20.293557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 906/1826] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:20 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:20.295561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae0a80-4dd6-462a-a05a-fb5f56526969 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:20.312735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae0a80-4dd6-462a-a05a-fb5f56526969 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:20.440951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48565dd9-89e4-4be3-8e7f-24c3f96c359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:20.441565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48565dd9-89e4-4be3-8e7f-24c3f96c359f admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:20.442389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48565dd9-89e4-4be3-8e7f-24c3f96c359f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:20.442389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48565dd9-89e4-4be3-8e7f-24c3f96c359f admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:20.442815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48565dd9-89e4-4be3-8e7f-24c3f96c359f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:20.447459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48565dd9-89e4-4be3-8e7f-24c3f96c359f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:20.524365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec1cab-a432-4d9f-b364-50bd8a862828 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:20.526094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b8ec1cab-a432-4d9f-b364-50bd8a862828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:20.539321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec1cab-a432-4d9f-b364-50bd8a862828 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:20.544411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:20.545074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:20.545354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:20.545626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] PATH_INFO: `/v3/users/14f8aea0e3574909801c2e4c8ad0e698` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:20.546863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=14f8aea0e3574909801c2e4c8ad0e698)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:20.548784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:20.556568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:20.564773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 921/1827] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:20 2022] POST /identity/v3/users => generated 294 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:20.567852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48565dd9-89e4-4be3-8e7f-24c3f96c359f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:20.582014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48565dd9-89e4-4be3-8e7f-24c3f96c359f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:20.586811 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cfdb9-9914-489e-8636-9cfe274d09a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:20.587651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cfdb9-9914-489e-8636-9cfe274d09a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:20.587823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cfdb9-9914-489e-8636-9cfe274d09a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:20.588148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cfdb9-9914-489e-8636-9cfe274d09a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:20.588812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cfdb9-9914-489e-8636-9cfe274d09a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:20.590462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cfdb9-9914-489e-8636-9cfe274d09a7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:20.602201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 922/1828] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:20.606389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cfdb9-9914-489e-8636-9cfe274d09a7 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:20.616814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cfdb9-9914-489e-8636-9cfe274d09a7 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:20.626044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfc508-3190-460c-b855-a853a4c6c9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:20.628026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfc508-3190-460c-b855-a853a4c6c9af admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:20.628026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfc508-3190-460c-b855-a853a4c6c9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:20.628026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfc508-3190-460c-b855-a853a4c6c9af admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:20.628026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfc508-3190-460c-b855-a853a4c6c9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:20.629664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfc508-3190-460c-b855-a853a4c6c9af admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:20.641992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 923/1829] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:20.645820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfc508-3190-460c-b855-a853a4c6c9af admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:20.655720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfc508-3190-460c-b855-a853a4c6c9af admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:20.660788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16855ea5-30fa-47e8-8411-f3616f38a8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:20.661441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16855ea5-30fa-47e8-8411-f3616f38a8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:20.661723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16855ea5-30fa-47e8-8411-f3616f38a8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:20.662049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16855ea5-30fa-47e8-8411-f3616f38a8f7 admin admin] PATH_INFO: `/v3/projects/bc3a70c3204f478ab16becbf47d40779/users/2ffddf0618594069b6c780f8700a16bc/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:20.680426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16855ea5-30fa-47e8-8411-f3616f38a8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3a70c3204f478ab16becbf47d40779, user_id=2ffddf0618594069b6c780f8700a16bc, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:20.681027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16855ea5-30fa-47e8-8411-f3616f38a8f7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:20.725484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14f8aea0e3574909801c2e4c8ad0e698'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:20.726827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14f8aea0e3574909801c2e4c8ad0e698'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:20.727601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14f8aea0e3574909801c2e4c8ad0e698'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:20.786516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 924/1830] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:20 2022] PUT /identity/v3/projects/bc3a70c3204f478ab16becbf47d40779/users/2ffddf0618594069b6c780f8700a16bc/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:20.798786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a5bb3-d343-4c5d-89bc-fa69a7eb713f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:20.799078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a5bb3-d343-4c5d-89bc-fa69a7eb713f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:20.799355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a5bb3-d343-4c5d-89bc-fa69a7eb713f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:20.826691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14f8aea0e3574909801c2e4c8ad0e698'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:20.835405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9195504c-c250-4f1e-9231-a97f5e128c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14f8aea0e3574909801c2e4c8ad0e698'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:20.851971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-657a5bb3-d343-4c5d-89bc-fa69a7eb713f None None] MFA Rules not processed for user `2ffddf0618594069b6c780f8700a16bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:20.853256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-657a5bb3-d343-4c5d-89bc-fa69a7eb713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:20.860971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 907/1831] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:20 2022] DELETE /identity/v3/users/14f8aea0e3574909801c2e4c8ad0e698 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:20.868356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85a058-8125-4b8e-b856-528ef97722f1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:20.868899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85a058-8125-4b8e-b856-528ef97722f1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:20.869401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85a058-8125-4b8e-b856-528ef97722f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:20.897074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8b85a058-8125-4b8e-b856-528ef97722f1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:20.898671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8b85a058-8125-4b8e-b856-528ef97722f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:20.916520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:20.943709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 925/1832] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:20.947833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a5bb3-d343-4c5d-89bc-fa69a7eb713f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:20.950664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-657a5bb3-d343-4c5d-89bc-fa69a7eb713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:20.977580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 908/1833] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:20.987674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85a058-8125-4b8e-b856-528ef97722f1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:20.990161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8b85a058-8125-4b8e-b856-528ef97722f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:21.028751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a5bb3-d343-4c5d-89bc-fa69a7eb713f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.033690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097183e7-f54e-42b6-9ab0-1fb91b04a397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.034415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097183e7-f54e-42b6-9ab0-1fb91b04a397 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.034768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097183e7-f54e-42b6-9ab0-1fb91b04a397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.035111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097183e7-f54e-42b6-9ab0-1fb91b04a397 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.035814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097183e7-f54e-42b6-9ab0-1fb91b04a397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.038045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097183e7-f54e-42b6-9ab0-1fb91b04a397 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.055026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85a058-8125-4b8e-b856-528ef97722f1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.058853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a0718-d17d-4f4e-b4ef-79531290d8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.059371 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a0718-d17d-4f4e-b4ef-79531290d8fb service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.059668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a0718-d17d-4f4e-b4ef-79531290d8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.059934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a0718-d17d-4f4e-b4ef-79531290d8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.069462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a0718-d17d-4f4e-b4ef-79531290d8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.070918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a0718-d17d-4f4e-b4ef-79531290d8fb service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.080545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 909/1834] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:21.081366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a0718-d17d-4f4e-b4ef-79531290d8fb service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:21.083033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-771a0718-d17d-4f4e-b4ef-79531290d8fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:21.096980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a0718-d17d-4f4e-b4ef-79531290d8fb service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.102988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.103690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.103916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.104188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] PATH_INFO: `/v3/users/c3f6348ab76c440abea1a1bf0a5b276f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.105496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f6348ab76c440abea1a1bf0a5b276f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.108212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.135068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 926/1835] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:20 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:21.138979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097183e7-f54e-42b6-9ab0-1fb91b04a397 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:21.204638 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097183e7-f54e-42b6-9ab0-1fb91b04a397 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.209966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae861d-2d54-4b1d-b895-73c9d98c2745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.210577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae861d-2d54-4b1d-b895-73c9d98c2745 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.210853 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae861d-2d54-4b1d-b895-73c9d98c2745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.211147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae861d-2d54-4b1d-b895-73c9d98c2745 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.211780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae861d-2d54-4b1d-b895-73c9d98c2745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.213705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae861d-2d54-4b1d-b895-73c9d98c2745 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.306485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 927/1836] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:21 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:21.312149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae861d-2d54-4b1d-b895-73c9d98c2745 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:21.332011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae861d-2d54-4b1d-b895-73c9d98c2745 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.339405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f6348ab76c440abea1a1bf0a5b276f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.339405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f6348ab76c440abea1a1bf0a5b276f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.339405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f6348ab76c440abea1a1bf0a5b276f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.340044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51444dc3-9c41-42ef-a9bd-f1f945c5430b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.340044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51444dc3-9c41-42ef-a9bd-f1f945c5430b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.340044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51444dc3-9c41-42ef-a9bd-f1f945c5430b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.340044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51444dc3-9c41-42ef-a9bd-f1f945c5430b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.340349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51444dc3-9c41-42ef-a9bd-f1f945c5430b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.341216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51444dc3-9c41-42ef-a9bd-f1f945c5430b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.368252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 928/1837] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:21.368252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:21.368252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51444dc3-9c41-42ef-a9bd-f1f945c5430b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:21.368252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-51444dc3-9c41-42ef-a9bd-f1f945c5430b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:21.378740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51444dc3-9c41-42ef-a9bd-f1f945c5430b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.383711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ae233-a769-4a25-84d7-2976b20ee8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.384431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ae233-a769-4a25-84d7-2976b20ee8ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.384925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ae233-a769-4a25-84d7-2976b20ee8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.385205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ae233-a769-4a25-84d7-2976b20ee8ca admin admin] PATH_INFO: `/v3/projects/e96846edffd74e319ca177f5abc33adf/users/da748245c4a14b77a7f4f8a7318ccd30/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.402606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ae233-a769-4a25-84d7-2976b20ee8ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96846edffd74e319ca177f5abc33adf, user_id=da748245c4a14b77a7f4f8a7318ccd30, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.405447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ae233-a769-4a25-84d7-2976b20ee8ca admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.417706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f6348ab76c440abea1a1bf0a5b276f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.428934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68e4168f-d8f4-4cae-bc8f-0eca2f11cd61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f6348ab76c440abea1a1bf0a5b276f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.477239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 910/1838] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:21 2022] DELETE /identity/v3/users/c3f6348ab76c440abea1a1bf0a5b276f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:21.481650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be86c9-b5e9-4040-8eb9-5003b6ea3efd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.482037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be86c9-b5e9-4040-8eb9-5003b6ea3efd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.482759 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be86c9-b5e9-4040-8eb9-5003b6ea3efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.492960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 929/1839] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:21 2022] PUT /identity/v3/projects/e96846edffd74e319ca177f5abc33adf/users/da748245c4a14b77a7f4f8a7318ccd30/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:21.503258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ae233-a769-4a25-84d7-2976b20ee8ca admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:21.506424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-28be86c9-b5e9-4040-8eb9-5003b6ea3efd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:21.507968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-28be86c9-b5e9-4040-8eb9-5003b6ea3efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:21.601955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ae233-a769-4a25-84d7-2976b20ee8ca admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.608073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a846b89-8cb7-4b2a-a1b9-429264d17aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.609015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a846b89-8cb7-4b2a-a1b9-429264d17aba admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.609708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a846b89-8cb7-4b2a-a1b9-429264d17aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.610253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a846b89-8cb7-4b2a-a1b9-429264d17aba admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.610966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a846b89-8cb7-4b2a-a1b9-429264d17aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.612775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a846b89-8cb7-4b2a-a1b9-429264d17aba admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.616358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 911/1840] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:21.617322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be86c9-b5e9-4040-8eb9-5003b6ea3efd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:21.619357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-28be86c9-b5e9-4040-8eb9-5003b6ea3efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:21.628289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 930/1841] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:21.640157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be86c9-b5e9-4040-8eb9-5003b6ea3efd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.644767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a846b89-8cb7-4b2a-a1b9-429264d17aba admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:21.645985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.646632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.646926 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.647188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] PATH_INFO: `/v3/projects/b8fd58bb440d40bda9dc45610863470b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.648392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8fd58bb440d40bda9dc45610863470b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.656394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.678451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a846b89-8cb7-4b2a-a1b9-429264d17aba admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.693445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac11f4-fef6-4766-995d-7006659818cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.694974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac11f4-fef6-4766-995d-7006659818cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.697119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac11f4-fef6-4766-995d-7006659818cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.697119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac11f4-fef6-4766-995d-7006659818cc admin admin] PATH_INFO: `/v3/projects/e96846edffd74e319ca177f5abc33adf/users/da748245c4a14b77a7f4f8a7318ccd30/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.700252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac11f4-fef6-4766-995d-7006659818cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96846edffd74e319ca177f5abc33adf, user_id=da748245c4a14b77a7f4f8a7318ccd30, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.704169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac11f4-fef6-4766-995d-7006659818cc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.730320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 931/1842] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:21 2022] PUT /identity/v3/projects/e96846edffd74e319ca177f5abc33adf/users/da748245c4a14b77a7f4f8a7318ccd30/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:21.733741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac11f4-fef6-4766-995d-7006659818cc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:21.739380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7aac11f4-fef6-4766-995d-7006659818cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:21.822091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac11f4-fef6-4766-995d-7006659818cc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.830898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8fd58bb440d40bda9dc45610863470b', 'request_id': 'req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.839450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8fd58bb440d40bda9dc45610863470b', 'request_id': 'req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.846782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8fd58bb440d40bda9dc45610863470b', 'request_id': 'req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.890405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199c653-0441-4652-bd04-73abebfbbf57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.891512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199c653-0441-4652-bd04-73abebfbbf57 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.892034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199c653-0441-4652-bd04-73abebfbbf57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.892658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199c653-0441-4652-bd04-73abebfbbf57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.941380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] The token cache is being invalidate because project b8fd58bb440d40bda9dc45610863470b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:21.941844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.944159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1820f914-cdb2-4f73-8d44-0e5f64ea4deb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:21.950323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 912/1843] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:21 2022] DELETE /identity/v3/projects/b8fd58bb440d40bda9dc45610863470b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:21.950323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dda83-27e8-4cfa-81d2-5dbc5cfcd807 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.950323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dda83-27e8-4cfa-81d2-5dbc5cfcd807 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.950323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dda83-27e8-4cfa-81d2-5dbc5cfcd807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.978174 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199c653-0441-4652-bd04-73abebfbbf57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.979937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199c653-0441-4652-bd04-73abebfbbf57 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.994468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 932/1844] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:21.995376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199c653-0441-4652-bd04-73abebfbbf57 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:22.007190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199c653-0441-4652-bd04-73abebfbbf57 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:22.015190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:22.016478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:22.017148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:22.018534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] PATH_INFO: `/v3/users/38d631eaee7c42538876cb4911598119` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:22.019741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=38d631eaee7c42538876cb4911598119)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:22.021216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:22.025687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7b2dda83-27e8-4cfa-81d2-5dbc5cfcd807 None None] MFA Rules not processed for user `da748245c4a14b77a7f4f8a7318ccd30`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:22.027089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7b2dda83-27e8-4cfa-81d2-5dbc5cfcd807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:22.139070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 913/1845] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:22.150285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dda83-27e8-4cfa-81d2-5dbc5cfcd807 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:22.151656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7b2dda83-27e8-4cfa-81d2-5dbc5cfcd807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:22.210041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dda83-27e8-4cfa-81d2-5dbc5cfcd807 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:22.213709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc56998-5888-456c-930d-5a7a27501d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:22.214391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc56998-5888-456c-930d-5a7a27501d09 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:22.214668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc56998-5888-456c-930d-5a7a27501d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:22.214941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc56998-5888-456c-930d-5a7a27501d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:22.224911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc56998-5888-456c-930d-5a7a27501d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:22.226362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc56998-5888-456c-930d-5a7a27501d09 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:22.236453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 914/1846] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:22.310917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38d631eaee7c42538876cb4911598119'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.311566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38d631eaee7c42538876cb4911598119'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.312196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38d631eaee7c42538876cb4911598119'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.320856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38d631eaee7c42538876cb4911598119'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.336383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38d631eaee7c42538876cb4911598119'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.363837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:22.430287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 933/1847] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:21 2022] DELETE /identity/v3/users/38d631eaee7c42538876cb4911598119 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:22.560148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc56998-5888-456c-930d-5a7a27501d09 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:22.566189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cdc56998-5888-456c-930d-5a7a27501d09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:22.582677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc56998-5888-456c-930d-5a7a27501d09 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:22.659154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:22.660090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:22.660090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:22.660319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] PATH_INFO: `/v3/projects/091d52f2209a4a79a1f5f8f27db8ebb5` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:22.661517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=091d52f2209a4a79a1f5f8f27db8ebb5)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:22.663579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:22.776399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '091d52f2209a4a79a1f5f8f27db8ebb5', 'request_id': 'req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.783148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '091d52f2209a4a79a1f5f8f27db8ebb5', 'request_id': 'req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.793569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '091d52f2209a4a79a1f5f8f27db8ebb5', 'request_id': 'req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.886922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] The token cache is being invalidate because project 091d52f2209a4a79a1f5f8f27db8ebb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:22.887572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.889425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:22.889716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 915/1848] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:22 2022] DELETE /identity/v3/projects/091d52f2209a4a79a1f5f8f27db8ebb5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:22.893745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:22.895746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:22.917659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dda57-de44-44d1-8c45-b8995b954e9a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:23.004324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:23.004987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:23.005292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:23.005536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] PATH_INFO: `/v3/users/290d77d1a52b4d45a314987bcad1efa2` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:23.006763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] RBAC: Authorizing `identity:delete_user(user_id=290d77d1a52b4d45a314987bcad1efa2)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:23.008895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:23.074140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:23.075674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:23.089178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25f3fc-4d43-4e05-9a04-c3bd9b4f4a1a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:23.095705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:23.096840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:23.102023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:23.102023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] PATH_INFO: `/v3/projects/55f37a172b6c49048890033f36548342` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:23.102023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f37a172b6c49048890033f36548342)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:23.102023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:23.191936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290d77d1a52b4d45a314987bcad1efa2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.192525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290d77d1a52b4d45a314987bcad1efa2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.193141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290d77d1a52b4d45a314987bcad1efa2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.201241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290d77d1a52b4d45a314987bcad1efa2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.213805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a10c1874-5cd3-4254-9e16-1b5d2efb351d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290d77d1a52b4d45a314987bcad1efa2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.234984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f37a172b6c49048890033f36548342', 'request_id': 'req-bd6edf69-a11e-4d64-8d15-e07377a01f96'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.241286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f37a172b6c49048890033f36548342', 'request_id': 'req-bd6edf69-a11e-4d64-8d15-e07377a01f96'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.248919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f37a172b6c49048890033f36548342', 'request_id': 'req-bd6edf69-a11e-4d64-8d15-e07377a01f96'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.286824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:23.314937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 934/1849] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:22 2022] DELETE /identity/v3/users/290d77d1a52b4d45a314987bcad1efa2 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:23.396457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] The token cache is being invalidate because project 55f37a172b6c49048890033f36548342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:23.396997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.397843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd6edf69-a11e-4d64-8d15-e07377a01f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:23.399990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 916/1850] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:23 2022] DELETE /identity/v3/projects/55f37a172b6c49048890033f36548342 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:23.405536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bdd1b-8b18-4e40-aab4-979843459d5b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:23.406190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bdd1b-8b18-4e40-aab4-979843459d5b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:23.406657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bdd1b-8b18-4e40-aab4-979843459d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:23.428955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-692bdd1b-8b18-4e40-aab4-979843459d5b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:23.430359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-692bdd1b-8b18-4e40-aab4-979843459d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:23.491287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 935/1851] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:23.499323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8ae21-b536-4930-9ef7-0a96df818038 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:23.499670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8ae21-b536-4930-9ef7-0a96df818038 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:23.500147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8ae21-b536-4930-9ef7-0a96df818038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:23.524427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-eae8ae21-b536-4930-9ef7-0a96df818038 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:23.525694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-eae8ae21-b536-4930-9ef7-0a96df818038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:23.535446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 917/1852] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:23.541271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bdd1b-8b18-4e40-aab4-979843459d5b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:23.556664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bdd1b-8b18-4e40-aab4-979843459d5b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:23.572719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:23.573713 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:23.574255 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:23.574745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:23.575635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:25:23.577549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:23.580094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:23.595727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 936/1853] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:25:23.599207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:23.606151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8ae21-b536-4930-9ef7-0a96df818038 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:23.616780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8ae21-b536-4930-9ef7-0a96df818038 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:23.625015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c065f82-8452-46b4-8001-8f285be2b8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:23.629028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c065f82-8452-46b4-8001-8f285be2b8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:23.629028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c065f82-8452-46b4-8001-8f285be2b8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:23.629028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c065f82-8452-46b4-8001-8f285be2b8c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:23.629028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c065f82-8452-46b4-8001-8f285be2b8c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:23.629746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c065f82-8452-46b4-8001-8f285be2b8c9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:23.725623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:23.727909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:23.749751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 918/1854] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:23 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:23.754265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c065f82-8452-46b4-8001-8f285be2b8c9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:23.839243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0629dd7-3954-4491-9add-0d60ece1626e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:23.848611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c065f82-8452-46b4-8001-8f285be2b8c9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:23.858571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a19fc-f3b7-46b5-b6e9-595583504b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:23.859192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a19fc-f3b7-46b5-b6e9-595583504b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:23.859468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a19fc-f3b7-46b5-b6e9-595583504b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:23.859738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a19fc-f3b7-46b5-b6e9-595583504b31 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:23.860383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a19fc-f3b7-46b5-b6e9-595583504b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:23.862335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a19fc-f3b7-46b5-b6e9-595583504b31 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:23.894511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b3860-2fd6-49a2-9c72-64e4639bf3c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:23.895148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3860-2fd6-49a2-9c72-64e4639bf3c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:23.895415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3860-2fd6-49a2-9c72-64e4639bf3c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:23.895689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3860-2fd6-49a2-9c72-64e4639bf3c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:23.897443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f56b3860-2fd6-49a2-9c72-64e4639bf3c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:23.957252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b3860-2fd6-49a2-9c72-64e4639bf3c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:23.961941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b3860-2fd6-49a2-9c72-64e4639bf3c2 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:23.968893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 937/1855] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:23.990486 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 919/1856] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:23 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:23.992997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b3860-2fd6-49a2-9c72-64e4639bf3c2 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.004962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a19fc-f3b7-46b5-b6e9-595583504b31 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.008764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1b6a19fc-f3b7-46b5-b6e9-595583504b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:24.016771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b3860-2fd6-49a2-9c72-64e4639bf3c2 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.031268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a19fc-f3b7-46b5-b6e9-595583504b31 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.031935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf77db-eb6a-49e5-be12-f7b5693c238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.032577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf77db-eb6a-49e5-be12-f7b5693c238d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.034136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf77db-eb6a-49e5-be12-f7b5693c238d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.034642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf77db-eb6a-49e5-be12-f7b5693c238d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.035333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf77db-eb6a-49e5-be12-f7b5693c238d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.037154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf77db-eb6a-49e5-be12-f7b5693c238d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.046587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.047250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.047535 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.047820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] PATH_INFO: `/v3/projects/66f49554fcc54288a9a4ca4d1b38e5ba` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.050917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f49554fcc54288a9a4ca4d1b38e5ba)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.054261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.057294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 938/1857] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:24.062479 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf77db-eb6a-49e5-be12-f7b5693c238d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.076751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf77db-eb6a-49e5-be12-f7b5693c238d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.084746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cca40b-e06f-4931-8a66-20542cca975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.085431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cca40b-e06f-4931-8a66-20542cca975f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.085723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cca40b-e06f-4931-8a66-20542cca975f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.086038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cca40b-e06f-4931-8a66-20542cca975f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.087216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cca40b-e06f-4931-8a66-20542cca975f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.088646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cca40b-e06f-4931-8a66-20542cca975f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.102769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 939/1858] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:24.106862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cca40b-e06f-4931-8a66-20542cca975f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.118737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cca40b-e06f-4931-8a66-20542cca975f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.124122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8f958-264d-44a0-acac-f658884fc893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.124795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8f958-264d-44a0-acac-f658884fc893 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.125157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8f958-264d-44a0-acac-f658884fc893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.125438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8f958-264d-44a0-acac-f658884fc893 admin admin] PATH_INFO: `/v3/projects/80d14e23390b4eefaf754731aa861d76/users/31aa284b5e9e436ba4b17c48d1bdd10d/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.142465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8f958-264d-44a0-acac-f658884fc893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d14e23390b4eefaf754731aa861d76, user_id=31aa284b5e9e436ba4b17c48d1bdd10d, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.145204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8f958-264d-44a0-acac-f658884fc893 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.246449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 940/1859] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:24 2022] PUT /identity/v3/projects/80d14e23390b4eefaf754731aa861d76/users/31aa284b5e9e436ba4b17c48d1bdd10d/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:24.253753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f49554fcc54288a9a4ca4d1b38e5ba', 'request_id': 'req-32e46748-dd0e-40d2-b327-1e9647e03fa4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:24.256995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f8f94-6caa-4a40-a24c-655dcbaddfce None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.257435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f8f94-6caa-4a40-a24c-655dcbaddfce None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.257774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f8f94-6caa-4a40-a24c-655dcbaddfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.259169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f49554fcc54288a9a4ca4d1b38e5ba', 'request_id': 'req-32e46748-dd0e-40d2-b327-1e9647e03fa4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:24.266396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f49554fcc54288a9a4ca4d1b38e5ba', 'request_id': 'req-32e46748-dd0e-40d2-b327-1e9647e03fa4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:24.292466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4b3f8f94-6caa-4a40-a24c-655dcbaddfce None None] MFA Rules not processed for user `31aa284b5e9e436ba4b17c48d1bdd10d`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:24.293783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4b3f8f94-6caa-4a40-a24c-655dcbaddfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:24.369466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] The token cache is being invalidate because project 66f49554fcc54288a9a4ca4d1b38e5ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:24.370631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:24.372016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32e46748-dd0e-40d2-b327-1e9647e03fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:24.374303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 941/1860] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:24.375204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 920/1861] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:24 2022] DELETE /identity/v3/projects/66f49554fcc54288a9a4ca4d1b38e5ba => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:24.382623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f8f94-6caa-4a40-a24c-655dcbaddfce None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.383844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad394c7d-1258-422e-8c8f-c01e18abad2e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.384042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad394c7d-1258-422e-8c8f-c01e18abad2e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.384318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad394c7d-1258-422e-8c8f-c01e18abad2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.389811 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:24.440833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ad394c7d-1258-422e-8c8f-c01e18abad2e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:24.444662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ad394c7d-1258-422e-8c8f-c01e18abad2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:24.541055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:24.548784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f8f94-6caa-4a40-a24c-655dcbaddfce None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.555449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b667f0-a36d-4920-b5f4-cca51e28fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.556267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b667f0-a36d-4920-b5f4-cca51e28fe5a admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.556697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b667f0-a36d-4920-b5f4-cca51e28fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.557124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b667f0-a36d-4920-b5f4-cca51e28fe5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.557768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b667f0-a36d-4920-b5f4-cca51e28fe5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.559937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b667f0-a36d-4920-b5f4-cca51e28fe5a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.565557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 921/1862] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:24.579475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002022-c7d9-4093-a491-6a2350d1e184 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.579858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002022-c7d9-4093-a491-6a2350d1e184 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.580138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002022-c7d9-4093-a491-6a2350d1e184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.601059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-44002022-c7d9-4093-a491-6a2350d1e184 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:24.602323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-44002022-c7d9-4093-a491-6a2350d1e184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:24.610964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 922/1863] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:24.615119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44002022-c7d9-4093-a491-6a2350d1e184 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.624517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44002022-c7d9-4093-a491-6a2350d1e184 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.629949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c3b6f-6890-49c0-afb5-61616b129396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.630551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c3b6f-6890-49c0-afb5-61616b129396 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.635853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c3b6f-6890-49c0-afb5-61616b129396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.635853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c3b6f-6890-49c0-afb5-61616b129396 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.635853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c3b6f-6890-49c0-afb5-61616b129396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:25:24.635853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c3b6f-6890-49c0-afb5-61616b129396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.635853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c3b6f-6890-49c0-afb5-61616b129396 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.645319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 923/1864] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:25:24.649586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c3b6f-6890-49c0-afb5-61616b129396 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.662143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c3b6f-6890-49c0-afb5-61616b129396 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.667788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f299f-13b9-44a0-9d08-09bbeabfdf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.668270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f299f-13b9-44a0-9d08-09bbeabfdf5a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.668536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f299f-13b9-44a0-9d08-09bbeabfdf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.668808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f299f-13b9-44a0-9d08-09bbeabfdf5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.669564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f299f-13b9-44a0-9d08-09bbeabfdf5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.672078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f299f-13b9-44a0-9d08-09bbeabfdf5a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.672815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 942/1865] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:24 2022] POST /identity/v3/projects => generated 377 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:24.676361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b667f0-a36d-4920-b5f4-cca51e28fe5a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.678045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d3b667f0-a36d-4920-b5f4-cca51e28fe5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:24.743279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b667f0-a36d-4920-b5f4-cca51e28fe5a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.753706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c17c9-dd61-4023-89d0-985df9dd1abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.754392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c17c9-dd61-4023-89d0-985df9dd1abf admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.755225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c17c9-dd61-4023-89d0-985df9dd1abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.765648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c17c9-dd61-4023-89d0-985df9dd1abf admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.765648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c17c9-dd61-4023-89d0-985df9dd1abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.765648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c17c9-dd61-4023-89d0-985df9dd1abf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.783811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 924/1866] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:24 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:24.785189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f299f-13b9-44a0-9d08-09bbeabfdf5a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.854071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f299f-13b9-44a0-9d08-09bbeabfdf5a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.860334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934f812-5084-439c-94bc-0079a0cfbcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.860956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934f812-5084-439c-94bc-0079a0cfbcae admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.861241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934f812-5084-439c-94bc-0079a0cfbcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.861519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934f812-5084-439c-94bc-0079a0cfbcae admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.862195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934f812-5084-439c-94bc-0079a0cfbcae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.864208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934f812-5084-439c-94bc-0079a0cfbcae admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.907295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 943/1867] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:24 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:24.910673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c17c9-dd61-4023-89d0-985df9dd1abf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.925613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c17c9-dd61-4023-89d0-985df9dd1abf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.931834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35865e-5c2a-4568-a6fc-40278b434020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.932544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35865e-5c2a-4568-a6fc-40278b434020 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.932885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35865e-5c2a-4568-a6fc-40278b434020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.933219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35865e-5c2a-4568-a6fc-40278b434020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.933895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35865e-5c2a-4568-a6fc-40278b434020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.935992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35865e-5c2a-4568-a6fc-40278b434020 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.949446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 944/1868] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:24.950826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35865e-5c2a-4568-a6fc-40278b434020 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.961752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35865e-5c2a-4568-a6fc-40278b434020 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.967566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f546d-327d-493b-b28b-f201d655743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:24.968347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f546d-327d-493b-b28b-f201d655743d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.968626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f546d-327d-493b-b28b-f201d655743d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.968929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f546d-327d-493b-b28b-f201d655743d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.969581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f546d-327d-493b-b28b-f201d655743d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:24.973063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f546d-327d-493b-b28b-f201d655743d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:24.982777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 945/1869] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:24.987163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f546d-327d-493b-b28b-f201d655743d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.997318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f546d-327d-493b-b28b-f201d655743d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.002492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37574cc-02cd-46b8-b2e1-f6e8514819b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.003254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37574cc-02cd-46b8-b2e1-f6e8514819b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.003330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37574cc-02cd-46b8-b2e1-f6e8514819b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.003963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37574cc-02cd-46b8-b2e1-f6e8514819b9 admin admin] PATH_INFO: `/v3/projects/c5b4e10bd9884a5b838ae4aaf7b575d7/users/26c8457f8528455499b104ceb8ada7f4/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.022056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37574cc-02cd-46b8-b2e1-f6e8514819b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b4e10bd9884a5b838ae4aaf7b575d7, user_id=26c8457f8528455499b104ceb8ada7f4, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.028151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 925/1870] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:24 2022] POST /identity/v3/users => generated 288 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:25.033432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934f812-5084-439c-94bc-0079a0cfbcae admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:25.035437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37574cc-02cd-46b8-b2e1-f6e8514819b9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.050776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934f812-5084-439c-94bc-0079a0cfbcae admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.056099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7baa79-7b5b-40eb-87a4-d6bd569e21b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.056829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7baa79-7b5b-40eb-87a4-d6bd569e21b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.057588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7baa79-7b5b-40eb-87a4-d6bd569e21b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.057588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7baa79-7b5b-40eb-87a4-d6bd569e21b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.058218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7baa79-7b5b-40eb-87a4-d6bd569e21b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.059902 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7baa79-7b5b-40eb-87a4-d6bd569e21b9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.075171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 926/1871] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:25:25.079954 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7baa79-7b5b-40eb-87a4-d6bd569e21b9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:25.089977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7baa79-7b5b-40eb-87a4-d6bd569e21b9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.095316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7b4b4-a4b3-46ce-9223-8a6121bc5567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.095939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7b4b4-a4b3-46ce-9223-8a6121bc5567 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.096222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7b4b4-a4b3-46ce-9223-8a6121bc5567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.096499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7b4b4-a4b3-46ce-9223-8a6121bc5567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.097183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7b4b4-a4b3-46ce-9223-8a6121bc5567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.098828 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7b4b4-a4b3-46ce-9223-8a6121bc5567 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.109620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 927/1872] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:25.111013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c214d00-8261-44e1-8b2a-b1329b25394e None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.111297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c214d00-8261-44e1-8b2a-b1329b25394e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.111571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c214d00-8261-44e1-8b2a-b1329b25394e None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.113238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 928/1873] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:25.113938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c214d00-8261-44e1-8b2a-b1329b25394e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:25.128473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c214d00-8261-44e1-8b2a-b1329b25394e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.132950 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd6742-cfef-423c-bef8-c1f5d8d4be80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.133583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd6742-cfef-423c-bef8-c1f5d8d4be80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.133859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd6742-cfef-423c-bef8-c1f5d8d4be80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.134161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd6742-cfef-423c-bef8-c1f5d8d4be80 admin admin] PATH_INFO: `/v3/projects/ce8c5144f1b440f2b33dbca10016d56e/users/3240f87bae9a448e9661e3228fb3e9c6/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.149064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd6742-cfef-423c-bef8-c1f5d8d4be80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8c5144f1b440f2b33dbca10016d56e, user_id=3240f87bae9a448e9661e3228fb3e9c6, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.151757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd6742-cfef-423c-bef8-c1f5d8d4be80 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.173757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 946/1874] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:24 2022] PUT /identity/v3/projects/c5b4e10bd9884a5b838ae4aaf7b575d7/users/26c8457f8528455499b104ceb8ada7f4/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:25.175805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713763a-596c-451d-b809-66f418f57668 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.176410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713763a-596c-451d-b809-66f418f57668 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.176871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713763a-596c-451d-b809-66f418f57668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.205123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5713763a-596c-451d-b809-66f418f57668 None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:25.206429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5713763a-596c-451d-b809-66f418f57668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:25.258913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 929/1875] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:25 2022] PUT /identity/v3/projects/ce8c5144f1b440f2b33dbca10016d56e/users/3240f87bae9a448e9661e3228fb3e9c6/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:25.260704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa900a-633a-4940-a43f-d46c18b06738 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.261063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa900a-633a-4940-a43f-d46c18b06738 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.261410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa900a-633a-4940-a43f-d46c18b06738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.288809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 947/1876] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:25:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:25.290443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d117b02-c790-4842-ac2b-86378f0bbd81 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.290937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d117b02-c790-4842-ac2b-86378f0bbd81 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.291217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d117b02-c790-4842-ac2b-86378f0bbd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.321511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8d117b02-c790-4842-ac2b-86378f0bbd81 None None] MFA Rules not processed for user `3240f87bae9a448e9661e3228fb3e9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:25.327036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8d117b02-c790-4842-ac2b-86378f0bbd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:25.334385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-69aa900a-633a-4940-a43f-d46c18b06738 None None] MFA Rules not processed for user `26c8457f8528455499b104ceb8ada7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:25.335641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-69aa900a-633a-4940-a43f-d46c18b06738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:25.395338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 948/1877] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:25.396549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d117b02-c790-4842-ac2b-86378f0bbd81 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:25.401567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8d117b02-c790-4842-ac2b-86378f0bbd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:25.440346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:25.463333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 930/1878] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:25.464154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa900a-633a-4940-a43f-d46c18b06738 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:25.512304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d117b02-c790-4842-ac2b-86378f0bbd81 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.518155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb63799-6c0f-43f8-b2a6-a3fae363c307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.518780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb63799-6c0f-43f8-b2a6-a3fae363c307 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.519064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb63799-6c0f-43f8-b2a6-a3fae363c307 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.519344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb63799-6c0f-43f8-b2a6-a3fae363c307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.529445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb63799-6c0f-43f8-b2a6-a3fae363c307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.531787 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb63799-6c0f-43f8-b2a6-a3fae363c307 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.547364 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 949/1879] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:25.548828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb63799-6c0f-43f8-b2a6-a3fae363c307 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:25.564532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb63799-6c0f-43f8-b2a6-a3fae363c307 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.568626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa900a-633a-4940-a43f-d46c18b06738 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.569573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d14320-e29e-486c-bef3-65db41b6eee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.570390 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d14320-e29e-486c-bef3-65db41b6eee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.570673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d14320-e29e-486c-bef3-65db41b6eee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.570941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d14320-e29e-486c-bef3-65db41b6eee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.571662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d14320-e29e-486c-bef3-65db41b6eee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.573603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d14320-e29e-486c-bef3-65db41b6eee8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.578348 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b38c3-d7b5-4d7e-bb05-6bdf7cc43040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.578976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b38c3-d7b5-4d7e-bb05-6bdf7cc43040 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.579260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b38c3-d7b5-4d7e-bb05-6bdf7cc43040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.579536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b38c3-d7b5-4d7e-bb05-6bdf7cc43040 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.580174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b38c3-d7b5-4d7e-bb05-6bdf7cc43040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.582638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b38c3-d7b5-4d7e-bb05-6bdf7cc43040 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.675672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 950/1880] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:25 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:25.679710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d14320-e29e-486c-bef3-65db41b6eee8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:25.692228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:25.715609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 931/1881] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:25:25 2022] POST /identity/v3/projects => generated 377 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:25.719083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b38c3-d7b5-4d7e-bb05-6bdf7cc43040 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:25.785632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d14320-e29e-486c-bef3-65db41b6eee8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.809227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:25.816295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b38c3-d7b5-4d7e-bb05-6bdf7cc43040 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.821506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe75b-81d7-470f-9f69-076df7e95c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.822212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe75b-81d7-470f-9f69-076df7e95c37 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.822485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe75b-81d7-470f-9f69-076df7e95c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.822752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe75b-81d7-470f-9f69-076df7e95c37 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.823380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfe75b-81d7-470f-9f69-076df7e95c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.825542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfe75b-81d7-470f-9f69-076df7e95c37 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.866307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87549ef3-2722-4150-a370-e66e01cada5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.866928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87549ef3-2722-4150-a370-e66e01cada5c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.867206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87549ef3-2722-4150-a370-e66e01cada5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.867481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87549ef3-2722-4150-a370-e66e01cada5c admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.868155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87549ef3-2722-4150-a370-e66e01cada5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.870177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87549ef3-2722-4150-a370-e66e01cada5c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.964099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 932/1882] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:25 2022] POST /identity/v3/users => generated 295 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:25:25.965233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe75b-81d7-470f-9f69-076df7e95c37 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:25.975803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe75b-81d7-470f-9f69-076df7e95c37 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:25.980462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01373874-c6f0-43e2-8c8c-64c334ce2032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.981064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01373874-c6f0-43e2-8c8c-64c334ce2032 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.981335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01373874-c6f0-43e2-8c8c-64c334ce2032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.981603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01373874-c6f0-43e2-8c8c-64c334ce2032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.985444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01373874-c6f0-43e2-8c8c-64c334ce2032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.985444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01373874-c6f0-43e2-8c8c-64c334ce2032 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.994888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 933/1883] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:25.999933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01373874-c6f0-43e2-8c8c-64c334ce2032 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.009954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 951/1884] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:25:25 2022] POST /identity/v3/users => generated 296 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:25:26.010973 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87549ef3-2722-4150-a370-e66e01cada5c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.013431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01373874-c6f0-43e2-8c8c-64c334ce2032 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.019700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a3d06-4fd4-4c40-8dc3-91e8635497f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.020437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a3d06-4fd4-4c40-8dc3-91e8635497f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.020722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a3d06-4fd4-4c40-8dc3-91e8635497f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.021020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a3d06-4fd4-4c40-8dc3-91e8635497f7 admin admin] PATH_INFO: `/v3/projects/826d165304764abc8b879fa4e5273fa9/users/b96e86ac3b28459bab1b4353d0ba018c/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.025081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87549ef3-2722-4150-a370-e66e01cada5c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.030399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1daae1-0aae-4805-856d-2209841f7d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.031008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1daae1-0aae-4805-856d-2209841f7d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.031288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1daae1-0aae-4805-856d-2209841f7d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.034470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1daae1-0aae-4805-856d-2209841f7d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.034470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1daae1-0aae-4805-856d-2209841f7d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.034470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1daae1-0aae-4805-856d-2209841f7d75 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.037606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a3d06-4fd4-4c40-8dc3-91e8635497f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=826d165304764abc8b879fa4e5273fa9, user_id=b96e86ac3b28459bab1b4353d0ba018c, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.040683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a3d06-4fd4-4c40-8dc3-91e8635497f7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.044454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 952/1885] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:26.049165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1daae1-0aae-4805-856d-2209841f7d75 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.059397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1daae1-0aae-4805-856d-2209841f7d75 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.064079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d6fca-731e-4e78-a9f5-d71ac9fe2411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.064763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d6fca-731e-4e78-a9f5-d71ac9fe2411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.065045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d6fca-731e-4e78-a9f5-d71ac9fe2411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.065320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d6fca-731e-4e78-a9f5-d71ac9fe2411 admin admin] PATH_INFO: `/v3/projects/9f395d71eacd495fab59c79800710621/users/d712bd12c6594c25803d5f10dc670ec0/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.080709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d6fca-731e-4e78-a9f5-d71ac9fe2411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f395d71eacd495fab59c79800710621, user_id=d712bd12c6594c25803d5f10dc670ec0, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.083399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d6fca-731e-4e78-a9f5-d71ac9fe2411 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.192637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 953/1886] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:26 2022] PUT /identity/v3/projects/9f395d71eacd495fab59c79800710621/users/d712bd12c6594c25803d5f10dc670ec0/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:26.195649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 934/1887] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:25 2022] PUT /identity/v3/projects/826d165304764abc8b879fa4e5273fa9/users/b96e86ac3b28459bab1b4353d0ba018c/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:26.196533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d6fca-731e-4e78-a9f5-d71ac9fe2411 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.199481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a3d06-4fd4-4c40-8dc3-91e8635497f7 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.261858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d6fca-731e-4e78-a9f5-d71ac9fe2411 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.263542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a3d06-4fd4-4c40-8dc3-91e8635497f7 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.267611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9df05c-4c3f-4d16-9a58-ecf64d895913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.268269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9df05c-4c3f-4d16-9a58-ecf64d895913 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.268544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9df05c-4c3f-4d16-9a58-ecf64d895913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.268816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9df05c-4c3f-4d16-9a58-ecf64d895913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.269232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823beb3c-bdc5-45ed-9010-ec4168fd03e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.269469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9df05c-4c3f-4d16-9a58-ecf64d895913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.269919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823beb3c-bdc5-45ed-9010-ec4168fd03e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.270183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823beb3c-bdc5-45ed-9010-ec4168fd03e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.270460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823beb3c-bdc5-45ed-9010-ec4168fd03e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.271142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9df05c-4c3f-4d16-9a58-ecf64d895913 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.271242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823beb3c-bdc5-45ed-9010-ec4168fd03e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.272920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823beb3c-bdc5-45ed-9010-ec4168fd03e5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.281366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 954/1888] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:26.283469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 935/1889] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:25:26.286738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9df05c-4c3f-4d16-9a58-ecf64d895913 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.288416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823beb3c-bdc5-45ed-9010-ec4168fd03e5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.301164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9df05c-4c3f-4d16-9a58-ecf64d895913 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.303645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823beb3c-bdc5-45ed-9010-ec4168fd03e5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.306064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be65cac-a9ad-46d7-b949-a27302a02dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.306721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be65cac-a9ad-46d7-b949-a27302a02dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.307002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be65cac-a9ad-46d7-b949-a27302a02dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.307278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be65cac-a9ad-46d7-b949-a27302a02dd6 admin admin] PATH_INFO: `/v3/projects/9f395d71eacd495fab59c79800710621/users/d712bd12c6594c25803d5f10dc670ec0/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.308860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd94923-d4c0-4c6c-b7ea-4fb060cd715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.309602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd94923-d4c0-4c6c-b7ea-4fb060cd715b admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.309845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be65cac-a9ad-46d7-b949-a27302a02dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f395d71eacd495fab59c79800710621, user_id=d712bd12c6594c25803d5f10dc670ec0, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.309989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd94923-d4c0-4c6c-b7ea-4fb060cd715b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.310174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd94923-d4c0-4c6c-b7ea-4fb060cd715b admin admin] PATH_INFO: `/v3/projects/826d165304764abc8b879fa4e5273fa9/users/b96e86ac3b28459bab1b4353d0ba018c/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.313327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd94923-d4c0-4c6c-b7ea-4fb060cd715b admin admin] RBAC: Authorizing `identity:create_grant(project_id=826d165304764abc8b879fa4e5273fa9, user_id=b96e86ac3b28459bab1b4353d0ba018c, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.313685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be65cac-a9ad-46d7-b949-a27302a02dd6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.316600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd94923-d4c0-4c6c-b7ea-4fb060cd715b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.335530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 955/1890] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:26 2022] PUT /identity/v3/projects/9f395d71eacd495fab59c79800710621/users/d712bd12c6594c25803d5f10dc670ec0/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:25:26.337767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 936/1891] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:25:26 2022] PUT /identity/v3/projects/826d165304764abc8b879fa4e5273fa9/users/b96e86ac3b28459bab1b4353d0ba018c/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:26.347734 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd5c20-f683-4e45-a424-189225cc6dda None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.347931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3087f-87ee-4ea2-8585-4b69a8ce9892 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.348026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd5c20-f683-4e45-a424-189225cc6dda None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.348126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3087f-87ee-4ea2-8585-4b69a8ce9892 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.348286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd5c20-f683-4e45-a424-189225cc6dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.348383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3087f-87ee-4ea2-8585-4b69a8ce9892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.377692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4fe3087f-87ee-4ea2-8585-4b69a8ce9892 None None] MFA Rules not processed for user `b96e86ac3b28459bab1b4353d0ba018c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:26.377935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a9bd5c20-f683-4e45-a424-189225cc6dda None None] MFA Rules not processed for user `d712bd12c6594c25803d5f10dc670ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:26.378959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a9bd5c20-f683-4e45-a424-189225cc6dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:26.379076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4fe3087f-87ee-4ea2-8585-4b69a8ce9892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:26.450315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 937/1892] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:26.456936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 956/1893] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:26.462579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3087f-87ee-4ea2-8585-4b69a8ce9892 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.464381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4fe3087f-87ee-4ea2-8585-4b69a8ce9892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:26.469496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd5c20-f683-4e45-a424-189225cc6dda None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.471054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a9bd5c20-f683-4e45-a424-189225cc6dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:26.532950 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3087f-87ee-4ea2-8585-4b69a8ce9892 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.536395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd5c20-f683-4e45-a424-189225cc6dda None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.537076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c6331-a3d8-496e-ae52-32c5472ad09f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.537682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c6331-a3d8-496e-ae52-32c5472ad09f service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.540033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c6331-a3d8-496e-ae52-32c5472ad09f service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.544217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8c886-6352-420f-a8ea-04972481bb42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.544365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c6331-a3d8-496e-ae52-32c5472ad09f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.544441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8c886-6352-420f-a8ea-04972481bb42 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.544441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8c886-6352-420f-a8ea-04972481bb42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.544441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8c886-6352-420f-a8ea-04972481bb42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.553101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8c886-6352-420f-a8ea-04972481bb42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.554818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8c886-6352-420f-a8ea-04972481bb42 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.567816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 957/1894] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:26.600070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:26.601716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c6331-a3d8-496e-ae52-32c5472ad09f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.603237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c6331-a3d8-496e-ae52-32c5472ad09f service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.615198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 938/1895] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:26.754585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:26.764528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8c886-6352-420f-a8ea-04972481bb42 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.772193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8c886-6352-420f-a8ea-04972481bb42 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.776255 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba288c2c-a1c9-4b8e-bac6-4d18a95c4c63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.776871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba288c2c-a1c9-4b8e-bac6-4d18a95c4c63 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.777145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba288c2c-a1c9-4b8e-bac6-4d18a95c4c63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.777414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba288c2c-a1c9-4b8e-bac6-4d18a95c4c63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.779116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ba288c2c-a1c9-4b8e-bac6-4d18a95c4c63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:26.842525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba288c2c-a1c9-4b8e-bac6-4d18a95c4c63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.844217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba288c2c-a1c9-4b8e-bac6-4d18a95c4c63 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.865412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 958/1896] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:27.758625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:27.872384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:28.114520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:28.262690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:28.291642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:28.411359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:28.439844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:28.537360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:28.566342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:38.660438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=14 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:38.709017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=83 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:43.939228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c6331-a3d8-496e-ae52-32c5472ad09f service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:43.948286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c6331-a3d8-496e-ae52-32c5472ad09f service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:43.951986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe7fc-6d6b-404c-89d2-5410e1a4cb3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:43.953450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe7fc-6d6b-404c-89d2-5410e1a4cb3a service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:43.953734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe7fc-6d6b-404c-89d2-5410e1a4cb3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:43.954052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe7fc-6d6b-404c-89d2-5410e1a4cb3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:43.963654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbe7fc-6d6b-404c-89d2-5410e1a4cb3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:43.965093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbe7fc-6d6b-404c-89d2-5410e1a4cb3a service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:43.976745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 939/1897] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:50.354483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba288c2c-a1c9-4b8e-bac6-4d18a95c4c63 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:50.358820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba288c2c-a1c9-4b8e-bac6-4d18a95c4c63 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:50.362996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8736b-93cd-4943-93b7-e61ae86025f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:50.363670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8736b-93cd-4943-93b7-e61ae86025f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:50.363888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8736b-93cd-4943-93b7-e61ae86025f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:50.364164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8736b-93cd-4943-93b7-e61ae86025f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:50.366215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-14e8736b-93cd-4943-93b7-e61ae86025f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:50.431946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8736b-93cd-4943-93b7-e61ae86025f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:50.433407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8736b-93cd-4943-93b7-e61ae86025f9 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:50.443956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 959/1898] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:53.971249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:56.973745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe7fc-6d6b-404c-89d2-5410e1a4cb3a service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.060257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe7fc-6d6b-404c-89d2-5410e1a4cb3a service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.065778 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.071742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.071742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.071742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] PATH_INFO: `/v3/users/31aa284b5e9e436ba4b17c48d1bdd10d` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.071742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=31aa284b5e9e436ba4b17c48d1bdd10d)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.071742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.261522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31aa284b5e9e436ba4b17c48d1bdd10d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.262205 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31aa284b5e9e436ba4b17c48d1bdd10d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.262809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31aa284b5e9e436ba4b17c48d1bdd10d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.322950 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31aa284b5e9e436ba4b17c48d1bdd10d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.332721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31aa284b5e9e436ba4b17c48d1bdd10d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:57.352502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 940/1899] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:56 2022] DELETE /identity/v3/users/31aa284b5e9e436ba4b17c48d1bdd10d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:57.357149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dbe50-a6c5-41b9-a287-6d39f5f23f90 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.357621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dbe50-a6c5-41b9-a287-6d39f5f23f90 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.357709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dbe50-a6c5-41b9-a287-6d39f5f23f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.379304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2c0dbe50-a6c5-41b9-a287-6d39f5f23f90 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:57.379304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2c0dbe50-a6c5-41b9-a287-6d39f5f23f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:57.447011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 960/1900] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:57.457553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:57.459052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:57.520061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53196f1e-3dae-4bf0-962b-f1bd3be1ecdb admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:57.524033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef655818-8b0f-4084-aefc-66367c757c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:57.524783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef655818-8b0f-4084-aefc-66367c757c30 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:57.525123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef655818-8b0f-4084-aefc-66367c757c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:57.525410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef655818-8b0f-4084-aefc-66367c757c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:57.536178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef655818-8b0f-4084-aefc-66367c757c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:57.537653 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef655818-8b0f-4084-aefc-66367c757c30 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:57.548761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 941/1901] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:58.017805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0dbe50-a6c5-41b9-a287-6d39f5f23f90 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.019725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2c0dbe50-a6c5-41b9-a287-6d39f5f23f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:58.036452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0dbe50-a6c5-41b9-a287-6d39f5f23f90 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.042611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.043510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.043588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.044164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] PATH_INFO: `/v3/projects/80d14e23390b4eefaf754731aa861d76` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.046381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d14e23390b4eefaf754731aa861d76)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.048467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.209552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d14e23390b4eefaf754731aa861d76', 'request_id': 'req-93037f58-604b-4e91-9703-4b87528b8685'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.233696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d14e23390b4eefaf754731aa861d76', 'request_id': 'req-93037f58-604b-4e91-9703-4b87528b8685'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.344020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d14e23390b4eefaf754731aa861d76', 'request_id': 'req-93037f58-604b-4e91-9703-4b87528b8685'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.362910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] The token cache is being invalidate because project 80d14e23390b4eefaf754731aa861d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:25:58.363501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.364408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.366417 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 961/1902] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:58 2022] DELETE /identity/v3/projects/80d14e23390b4eefaf754731aa861d76 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:58.370828 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef655818-8b0f-4084-aefc-66367c757c30 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:58.386947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef655818-8b0f-4084-aefc-66367c757c30 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:58.424439 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:58.480971 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:58.482698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:58.482698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:58.482698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] PATH_INFO: `/v3/users/26c8457f8528455499b104ceb8ada7f4` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:58.484502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c8457f8528455499b104ceb8ada7f4)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:58.487038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:58.763326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c8457f8528455499b104ceb8ada7f4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.763937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c8457f8528455499b104ceb8ada7f4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.764701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c8457f8528455499b104ceb8ada7f4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.926124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c8457f8528455499b104ceb8ada7f4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.939089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c8457f8528455499b104ceb8ada7f4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:58.963192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 942/1903] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:25:58 2022] DELETE /identity/v3/users/26c8457f8528455499b104ceb8ada7f4 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:25:59.701528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:59.703621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:59.717494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93037f58-604b-4e91-9703-4b87528b8685 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:59.739544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:59.780930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:59.781784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:59.782166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:59.787351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] PATH_INFO: `/v3/projects/c5b4e10bd9884a5b838ae4aaf7b575d7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:59.787351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b4e10bd9884a5b838ae4aaf7b575d7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:59.787351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:59.917466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b4e10bd9884a5b838ae4aaf7b575d7', 'request_id': 'req-fbc328ef-3392-4c33-a112-6db53da401d5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:59.923280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b4e10bd9884a5b838ae4aaf7b575d7', 'request_id': 'req-fbc328ef-3392-4c33-a112-6db53da401d5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:25:59.988578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b4e10bd9884a5b838ae4aaf7b575d7', 'request_id': 'req-fbc328ef-3392-4c33-a112-6db53da401d5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.008301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] The token cache is being invalidate because project c5b4e10bd9884a5b838ae4aaf7b575d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:00.008594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.009280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.011436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 962/1904] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:25:59 2022] DELETE /identity/v3/projects/c5b4e10bd9884a5b838ae4aaf7b575d7 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:00.015895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:00.017368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:00.038289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab44f25-d9bb-4d37-889c-2ae81791ebd8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:00.130735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:00.138426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:00.138426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:00.138426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] PATH_INFO: `/v3/users/d712bd12c6594c25803d5f10dc670ec0` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:00.138426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d712bd12c6594c25803d5f10dc670ec0)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:00.138426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:00.264144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:00.266489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:00.283901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc328ef-3392-4c33-a112-6db53da401d5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:00.289653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:00.290657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:00.291089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:00.291344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] PATH_INFO: `/v3/users/21710b1a9c054c2bae5b48fc20d5d9b4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:00.293075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21710b1a9c054c2bae5b48fc20d5d9b4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:00.295299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:00.370145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:00.375677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd712bd12c6594c25803d5f10dc670ec0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.377184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd712bd12c6594c25803d5f10dc670ec0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.377814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd712bd12c6594c25803d5f10dc670ec0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.500068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd712bd12c6594c25803d5f10dc670ec0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.511180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd712bd12c6594c25803d5f10dc670ec0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.525720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21710b1a9c054c2bae5b48fc20d5d9b4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.526596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21710b1a9c054c2bae5b48fc20d5d9b4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.527505 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21710b1a9c054c2bae5b48fc20d5d9b4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.536495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21710b1a9c054c2bae5b48fc20d5d9b4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.537699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 943/1905] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:00 2022] DELETE /identity/v3/users/d712bd12c6594c25803d5f10dc670ec0 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:00.538875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:00.540470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:00.550867 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4e2e7ebf-398e-46a5-a260-d8045e26e1b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21710b1a9c054c2bae5b48fc20d5d9b4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.611681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c0288-6e76-4d14-b5c4-474a3a97dcdd admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:00.616589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:00.617227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:00.617507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:00.617779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] PATH_INFO: `/v3/users/0121ccb5874a447bbc2eef3beb139346` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:00.618955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0121ccb5874a447bbc2eef3beb139346)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:00.620915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:00.638115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 963/1906] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:00 2022] DELETE /identity/v3/users/21710b1a9c054c2bae5b48fc20d5d9b4 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:00.643114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56a679-ea66-4557-b57b-434c8b434385 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:00.643631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56a679-ea66-4557-b57b-434c8b434385 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:00.644206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56a679-ea66-4557-b57b-434c8b434385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:00.672706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1b56a679-ea66-4557-b57b-434c8b434385 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:00.674077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1b56a679-ea66-4557-b57b-434c8b434385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:00.746642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 964/1907] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:00.761713 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56a679-ea66-4557-b57b-434c8b434385 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:00.765276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1b56a679-ea66-4557-b57b-434c8b434385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:00.839060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56a679-ea66-4557-b57b-434c8b434385 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:00.843687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a4569-5965-4ded-b9a5-b3b255221ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:00.850448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a4569-5965-4ded-b9a5-b3b255221ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:00.850448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a4569-5965-4ded-b9a5-b3b255221ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:00.850448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a4569-5965-4ded-b9a5-b3b255221ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:00.856136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a4569-5965-4ded-b9a5-b3b255221ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:00.857965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a4569-5965-4ded-b9a5-b3b255221ec8 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:00.860815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0121ccb5874a447bbc2eef3beb139346'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.861606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0121ccb5874a447bbc2eef3beb139346'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.862353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0121ccb5874a447bbc2eef3beb139346'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.869996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 965/1908] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:00.914627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:00.937072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0121ccb5874a447bbc2eef3beb139346'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.945357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0121ccb5874a447bbc2eef3beb139346'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:00.966719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 944/1909] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:00 2022] DELETE /identity/v3/users/0121ccb5874a447bbc2eef3beb139346 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:00.971501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf50e1-eff2-4cd3-a994-11ea6f71a2ce None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:00.971792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf50e1-eff2-4cd3-a994-11ea6f71a2ce None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:00.972067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf50e1-eff2-4cd3-a994-11ea6f71a2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:00.991539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-16cf50e1-eff2-4cd3-a994-11ea6f71a2ce None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:00.992810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-16cf50e1-eff2-4cd3-a994-11ea6f71a2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:01.053027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 966/1910] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:01.066237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:01.068207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:01.151280 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc8bad-1842-4175-a02e-d1fb05bbbb58 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:01.158091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b75bc-00ba-4e07-8b28-c724da26ffa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:01.161547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b75bc-00ba-4e07-8b28-c724da26ffa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:01.161547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b75bc-00ba-4e07-8b28-c724da26ffa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:01.161547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b75bc-00ba-4e07-8b28-c724da26ffa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:01.162318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:01.175310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b75bc-00ba-4e07-8b28-c724da26ffa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:01.178318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b75bc-00ba-4e07-8b28-c724da26ffa9 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:01.190286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 945/1911] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:01.386731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf50e1-eff2-4cd3-a994-11ea6f71a2ce None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:01.388245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-16cf50e1-eff2-4cd3-a994-11ea6f71a2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:01.407527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf50e1-eff2-4cd3-a994-11ea6f71a2ce None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:01.417189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:01.418029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:01.418503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:01.418882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] PATH_INFO: `/v3/projects/3e00b6aa41da4f77af9e35c2d4e5c1ce` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:01.420024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e00b6aa41da4f77af9e35c2d4e5c1ce)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:01.422151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:01.451306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b75bc-00ba-4e07-8b28-c724da26ffa9 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:01.453287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c99b75bc-00ba-4e07-8b28-c724da26ffa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:01.480236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b75bc-00ba-4e07-8b28-c724da26ffa9 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:01.486519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:01.487359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:01.487877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:01.488228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] PATH_INFO: `/v3/projects/9f395d71eacd495fab59c79800710621` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:01.489459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f395d71eacd495fab59c79800710621)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:01.491418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:01.562499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e00b6aa41da4f77af9e35c2d4e5c1ce', 'request_id': 'req-9d68bce8-402b-4249-86ee-70262cb83a12'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.566612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e00b6aa41da4f77af9e35c2d4e5c1ce', 'request_id': 'req-9d68bce8-402b-4249-86ee-70262cb83a12'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.570937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:01.600736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f395d71eacd495fab59c79800710621', 'request_id': 'req-aa18e7a7-e010-468f-b60f-866a284b06e0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.608293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f395d71eacd495fab59c79800710621', 'request_id': 'req-aa18e7a7-e010-468f-b60f-866a284b06e0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.616266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f395d71eacd495fab59c79800710621', 'request_id': 'req-aa18e7a7-e010-468f-b60f-866a284b06e0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.657696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e00b6aa41da4f77af9e35c2d4e5c1ce', 'request_id': 'req-9d68bce8-402b-4249-86ee-70262cb83a12'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.672825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] The token cache is being invalidate because project 3e00b6aa41da4f77af9e35c2d4e5c1ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:01.673470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.674006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.678565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 967/1912] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:01 2022] DELETE /identity/v3/projects/3e00b6aa41da4f77af9e35c2d4e5c1ce => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:01.679762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:01.681228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:01.723443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] The token cache is being invalidate because project 9f395d71eacd495fab59c79800710621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:01.724288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.725327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aa18e7a7-e010-468f-b60f-866a284b06e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:01.734031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 946/1913] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:01 2022] DELETE /identity/v3/projects/9f395d71eacd495fab59c79800710621 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:01.736424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303ec0c-24ce-47e4-b3c3-7fea4bc7d3e8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:01.736821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303ec0c-24ce-47e4-b3c3-7fea4bc7d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:01.737227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303ec0c-24ce-47e4-b3c3-7fea4bc7d3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:01.771219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68bce8-402b-4249-86ee-70262cb83a12 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:01.793356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7303ec0c-24ce-47e4-b3c3-7fea4bc7d3e8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:01.798569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7303ec0c-24ce-47e4-b3c3-7fea4bc7d3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:01.896250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 947/1914] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:01.906373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42068c04-30cd-49ea-ad8c-10ebaae36b0d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:01.906741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:01.907004 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42068c04-30cd-49ea-ad8c-10ebaae36b0d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:01.907353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42068c04-30cd-49ea-ad8c-10ebaae36b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:01.907919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:01.908402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:01.908825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] PATH_INFO: `/v3/users/2ffddf0618594069b6c780f8700a16bc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:01.923093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffddf0618594069b6c780f8700a16bc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:01.934121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:01.942428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-42068c04-30cd-49ea-ad8c-10ebaae36b0d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:01.944064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-42068c04-30cd-49ea-ad8c-10ebaae36b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:01.954243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 948/1915] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:01.955307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42068c04-30cd-49ea-ad8c-10ebaae36b0d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:01.958821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-42068c04-30cd-49ea-ad8c-10ebaae36b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:01.984176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42068c04-30cd-49ea-ad8c-10ebaae36b0d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:01.990090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:01.990822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:01.991100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:01.991379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] PATH_INFO: `/v3/projects/9a205b5137a1474c82f08233653fa8d0` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:01.992527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a205b5137a1474c82f08233653fa8d0)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:01.994514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:02.124490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffddf0618594069b6c780f8700a16bc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.124746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a205b5137a1474c82f08233653fa8d0', 'request_id': 'req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.125518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffddf0618594069b6c780f8700a16bc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.126239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffddf0618594069b6c780f8700a16bc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.130264 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:02.134318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a205b5137a1474c82f08233653fa8d0', 'request_id': 'req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.134476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffddf0618594069b6c780f8700a16bc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.145549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a205b5137a1474c82f08233653fa8d0', 'request_id': 'req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.149141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffddf0618594069b6c780f8700a16bc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.234046 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] The token cache is being invalidate because project 9a205b5137a1474c82f08233653fa8d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:02.234345 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.235005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.240181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 949/1916] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:01 2022] DELETE /identity/v3/projects/9a205b5137a1474c82f08233653fa8d0 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:02.240181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.241358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:02.315139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5330db4-2dc8-4a05-b3d4-a79cb4dbf8ce admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.320374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:02.321008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.321291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.321569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.322274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:02.322587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:02.324367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:02.337706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 950/1917] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:26:02.338624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.340298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:02.353546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b469f8-a0bc-451c-8f21-cb01d1029971 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.358747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:02.359413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.359696 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.359974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] PATH_INFO: `/v3/users/1d77b799691d43f697f0b4cd2b032004` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.361188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d77b799691d43f697f0b4cd2b032004)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:02.363142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:02.377815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 968/1918] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:01 2022] DELETE /identity/v3/users/2ffddf0618594069b6c780f8700a16bc => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:02.377815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.378198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:02.402606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:02.460036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0337c5a-65e1-46ef-b6fa-f81f7315dbd6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.466153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728226d8-fb5d-41d4-a4b0-b4a86f83effa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:02.466948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728226d8-fb5d-41d4-a4b0-b4a86f83effa admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.467238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728226d8-fb5d-41d4-a4b0-b4a86f83effa admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.468262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728226d8-fb5d-41d4-a4b0-b4a86f83effa admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.468943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728226d8-fb5d-41d4-a4b0-b4a86f83effa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:02.470930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728226d8-fb5d-41d4-a4b0-b4a86f83effa admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:02.573486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 969/1919] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:02 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:02.578819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728226d8-fb5d-41d4-a4b0-b4a86f83effa admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.583589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d77b799691d43f697f0b4cd2b032004'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.584331 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d77b799691d43f697f0b4cd2b032004'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.584951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d77b799691d43f697f0b4cd2b032004'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.656794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728226d8-fb5d-41d4-a4b0-b4a86f83effa admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.657260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d77b799691d43f697f0b4cd2b032004'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.662114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbfed8-a303-4586-8fd4-0e3ded0da949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:02.662576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbfed8-a303-4586-8fd4-0e3ded0da949 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.663007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbfed8-a303-4586-8fd4-0e3ded0da949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.663291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbfed8-a303-4586-8fd4-0e3ded0da949 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.663935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbfed8-a303-4586-8fd4-0e3ded0da949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:02.666250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbfed8-a303-4586-8fd4-0e3ded0da949 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:02.670773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d77b799691d43f697f0b4cd2b032004'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:02.698243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 951/1920] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:02 2022] DELETE /identity/v3/users/1d77b799691d43f697f0b4cd2b032004 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:02.841052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 970/1921] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:02 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:02.847568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.849062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:02.868603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbfed8-a303-4586-8fd4-0e3ded0da949 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:02.870464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2ffbfed8-a303-4586-8fd4-0e3ded0da949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:02.890129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbfed8-a303-4586-8fd4-0e3ded0da949 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.965352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7294-63d9-4dff-997d-092287d028e4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:02.971159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f11115-3db0-48dc-b76c-50115e363aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:02.971924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f11115-3db0-48dc-b76c-50115e363aee admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:02.972600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f11115-3db0-48dc-b76c-50115e363aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:02.972953 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f11115-3db0-48dc-b76c-50115e363aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:02.973616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f11115-3db0-48dc-b76c-50115e363aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:02.975292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f11115-3db0-48dc-b76c-50115e363aee admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:02.991836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 952/1922] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:02.993080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f11115-3db0-48dc-b76c-50115e363aee admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:03.002905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:03.016367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f11115-3db0-48dc-b76c-50115e363aee admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:03.029742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196ba5d-ffc4-4d1a-96ee-3c66be7df20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:03.031434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ba5d-ffc4-4d1a-96ee-3c66be7df20d admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.031713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ba5d-ffc4-4d1a-96ee-3c66be7df20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.031986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ba5d-ffc4-4d1a-96ee-3c66be7df20d admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.032680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196ba5d-ffc4-4d1a-96ee-3c66be7df20d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:03.035287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196ba5d-ffc4-4d1a-96ee-3c66be7df20d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:03.040281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:03.041123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.041408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.044371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] PATH_INFO: `/v3/projects/bc3a70c3204f478ab16becbf47d40779` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.044371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3a70c3204f478ab16becbf47d40779)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:03.044829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:03.054572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 953/1923] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:03.056547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196ba5d-ffc4-4d1a-96ee-3c66be7df20d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:03.071435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196ba5d-ffc4-4d1a-96ee-3c66be7df20d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:03.079281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703c1e2-f414-4a2d-904f-50dd32bc55a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:03.084564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703c1e2-f414-4a2d-904f-50dd32bc55a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.084564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703c1e2-f414-4a2d-904f-50dd32bc55a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.084564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703c1e2-f414-4a2d-904f-50dd32bc55a7 admin admin] PATH_INFO: `/v3/projects/686d419304d44ecdaab67cf2aa926e94/users/d63295214c5f415eac699fbe4f0ed3b7/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.100235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703c1e2-f414-4a2d-904f-50dd32bc55a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=686d419304d44ecdaab67cf2aa926e94, user_id=d63295214c5f415eac699fbe4f0ed3b7, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:03.104504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703c1e2-f414-4a2d-904f-50dd32bc55a7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:03.236043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:03.236611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 954/1924] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:03 2022] PUT /identity/v3/projects/686d419304d44ecdaab67cf2aa926e94/users/d63295214c5f415eac699fbe4f0ed3b7/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:03.247320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3a70c3204f478ab16becbf47d40779', 'request_id': 'req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.247897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0ecc6-0d65-4c6e-a68a-4b5b80df51e6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.248180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0ecc6-0d65-4c6e-a68a-4b5b80df51e6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.248454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0ecc6-0d65-4c6e-a68a-4b5b80df51e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.253597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3a70c3204f478ab16becbf47d40779', 'request_id': 'req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.294666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-42b0ecc6-0d65-4c6e-a68a-4b5b80df51e6 None None] MFA Rules not processed for user `d63295214c5f415eac699fbe4f0ed3b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:03.296155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-42b0ecc6-0d65-4c6e-a68a-4b5b80df51e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:03.337687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3a70c3204f478ab16becbf47d40779', 'request_id': 'req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.356165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] The token cache is being invalidate because project bc3a70c3204f478ab16becbf47d40779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:03.356523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.357775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.362946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 971/1925] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:02 2022] DELETE /identity/v3/projects/bc3a70c3204f478ab16becbf47d40779 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:03.362946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:03.362946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:03.367276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 955/1926] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:03.368735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0ecc6-0d65-4c6e-a68a-4b5b80df51e6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:03.435897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0ecc6-0d65-4c6e-a68a-4b5b80df51e6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:03.441932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:03.442602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.442840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ae1c5-bca1-497f-a506-1f8514b2d8e9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:03.442939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.443139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] PATH_INFO: `/v3/users/da748245c4a14b77a7f4f8a7318ccd30` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.444339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da748245c4a14b77a7f4f8a7318ccd30)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:03.446317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:03.448005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:03.448642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.448931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.449205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] PATH_INFO: `/v3/projects/64ddde3cf4bf419ba4634151d1b9a7bb` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.450410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64ddde3cf4bf419ba4634151d1b9a7bb)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:03.452546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:03.712209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da748245c4a14b77a7f4f8a7318ccd30'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.713212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da748245c4a14b77a7f4f8a7318ccd30'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.714117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da748245c4a14b77a7f4f8a7318ccd30'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.715622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64ddde3cf4bf419ba4634151d1b9a7bb', 'request_id': 'req-6076c2d2-a760-4824-b723-f47581b32d62'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.722421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64ddde3cf4bf419ba4634151d1b9a7bb', 'request_id': 'req-6076c2d2-a760-4824-b723-f47581b32d62'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.780571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da748245c4a14b77a7f4f8a7318ccd30'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.783700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:03.790997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da748245c4a14b77a7f4f8a7318ccd30'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.811806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 956/1927] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:03 2022] DELETE /identity/v3/users/da748245c4a14b77a7f4f8a7318ccd30 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:03.812634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:03.814045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:03.875397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f20c93-5437-4bc2-95ee-4ac8f3de13f9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:03.877702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64ddde3cf4bf419ba4634151d1b9a7bb', 'request_id': 'req-6076c2d2-a760-4824-b723-f47581b32d62'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.879488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203baeb1-ccdc-47d8-ac2f-b1be53d4f352 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:03.880115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203baeb1-ccdc-47d8-ac2f-b1be53d4f352 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.880396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203baeb1-ccdc-47d8-ac2f-b1be53d4f352 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.880673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203baeb1-ccdc-47d8-ac2f-b1be53d4f352 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.883538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-203baeb1-ccdc-47d8-ac2f-b1be53d4f352 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:03.892824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] The token cache is being invalidate because project 64ddde3cf4bf419ba4634151d1b9a7bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:03.893136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.893979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6076c2d2-a760-4824-b723-f47581b32d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:03.895546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 972/1928] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:03 2022] DELETE /identity/v3/projects/64ddde3cf4bf419ba4634151d1b9a7bb => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:03.900516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af445b68-2655-46aa-af18-a036418e3f56 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:03.900807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af445b68-2655-46aa-af18-a036418e3f56 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:03.901087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af445b68-2655-46aa-af18-a036418e3f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:03.923217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-af445b68-2655-46aa-af18-a036418e3f56 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:03.924452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-af445b68-2655-46aa-af18-a036418e3f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:03.949128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203baeb1-ccdc-47d8-ac2f-b1be53d4f352 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:03.950677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203baeb1-ccdc-47d8-ac2f-b1be53d4f352 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:03.960108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 957/1929] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:03.978274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:03.990723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 973/1930] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:04.000038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbbf1e-65f9-4076-93ec-3db749b9dd1a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.000780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbbf1e-65f9-4076-93ec-3db749b9dd1a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.001373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbbf1e-65f9-4076-93ec-3db749b9dd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.024926 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-64dbbf1e-65f9-4076-93ec-3db749b9dd1a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:04.026131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-64dbbf1e-65f9-4076-93ec-3db749b9dd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.035605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 958/1931] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:04.043293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af445b68-2655-46aa-af18-a036418e3f56 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.055565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af445b68-2655-46aa-af18-a036418e3f56 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.064126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e95733-2405-4da0-8e02-c2cf5e4d9890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.067567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e95733-2405-4da0-8e02-c2cf5e4d9890 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.067567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e95733-2405-4da0-8e02-c2cf5e4d9890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.067567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e95733-2405-4da0-8e02-c2cf5e4d9890 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.067567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e95733-2405-4da0-8e02-c2cf5e4d9890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:04.067567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e95733-2405-4da0-8e02-c2cf5e4d9890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.068808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e95733-2405-4da0-8e02-c2cf5e4d9890 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.090360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 974/1932] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:26:04.091232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbbf1e-65f9-4076-93ec-3db749b9dd1a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.103615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbbf1e-65f9-4076-93ec-3db749b9dd1a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.117771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5afb9e-b95a-4240-90b9-45fcac2fb6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.119035 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5afb9e-b95a-4240-90b9-45fcac2fb6e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.119504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5afb9e-b95a-4240-90b9-45fcac2fb6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.120354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5afb9e-b95a-4240-90b9-45fcac2fb6e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.121156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5afb9e-b95a-4240-90b9-45fcac2fb6e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.123102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5afb9e-b95a-4240-90b9-45fcac2fb6e4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.237190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 959/1933] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:04.241380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e95733-2405-4da0-8e02-c2cf5e4d9890 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.251584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e95733-2405-4da0-8e02-c2cf5e4d9890 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.265256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5afb9e-b95a-4240-90b9-45fcac2fb6e4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.269490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0b5afb9e-b95a-4240-90b9-45fcac2fb6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.330297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90021a-50a5-4228-9ea7-0a99157f2c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.330950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90021a-50a5-4228-9ea7-0a99157f2c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.331262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90021a-50a5-4228-9ea7-0a99157f2c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.331544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90021a-50a5-4228-9ea7-0a99157f2c43 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.332184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90021a-50a5-4228-9ea7-0a99157f2c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.334168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90021a-50a5-4228-9ea7-0a99157f2c43 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.345092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:04.355152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5afb9e-b95a-4240-90b9-45fcac2fb6e4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.359816 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.360444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.360761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.361019 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] PATH_INFO: `/v3/projects/e96846edffd74e319ca177f5abc33adf` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.362270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96846edffd74e319ca177f5abc33adf)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.364226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.449896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 975/1934] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:04.456508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90021a-50a5-4228-9ea7-0a99157f2c43 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.462719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96846edffd74e319ca177f5abc33adf', 'request_id': 'req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.471379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96846edffd74e319ca177f5abc33adf', 'request_id': 'req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.471541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90021a-50a5-4228-9ea7-0a99157f2c43 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.476748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96846edffd74e319ca177f5abc33adf', 'request_id': 'req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.527139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629993d-191c-469f-a8b4-6339ce461039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.527747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629993d-191c-469f-a8b4-6339ce461039 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.528701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629993d-191c-469f-a8b4-6339ce461039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.528701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629993d-191c-469f-a8b4-6339ce461039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.528933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629993d-191c-469f-a8b4-6339ce461039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.530607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629993d-191c-469f-a8b4-6339ce461039 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.540652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 976/1935] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:04.544852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629993d-191c-469f-a8b4-6339ce461039 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.554976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629993d-191c-469f-a8b4-6339ce461039 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.559838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881aeb4-9fbc-4f48-ab79-823570dbe9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.563348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881aeb4-9fbc-4f48-ab79-823570dbe9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.563348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881aeb4-9fbc-4f48-ab79-823570dbe9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.563348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881aeb4-9fbc-4f48-ab79-823570dbe9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.563348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881aeb4-9fbc-4f48-ab79-823570dbe9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.563874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881aeb4-9fbc-4f48-ab79-823570dbe9cb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.576011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 977/1936] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:26:04.580813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881aeb4-9fbc-4f48-ab79-823570dbe9cb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.588522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] The token cache is being invalidate because project e96846edffd74e319ca177f5abc33adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:04.589133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.589967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-11b8a4d4-89e6-4a8d-8ee5-c419476d52c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:04.591711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 960/1937] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:04 2022] DELETE /identity/v3/projects/e96846edffd74e319ca177f5abc33adf => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:04.594338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881aeb4-9fbc-4f48-ab79-823570dbe9cb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.599678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f21380-7d21-4149-ad50-2e98cf57faf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.600339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f21380-7d21-4149-ad50-2e98cf57faf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.601092 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f21380-7d21-4149-ad50-2e98cf57faf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.601092 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f21380-7d21-4149-ad50-2e98cf57faf1 admin admin] PATH_INFO: `/v3/projects/a235ce07c1264370bdfb7d16112c9c3b/users/d763750b633e4c72baf7f7fb42fa983d/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.604527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3c452-da5d-454d-a245-be678c25083c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.604817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3c452-da5d-454d-a245-be678c25083c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.605094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3c452-da5d-454d-a245-be678c25083c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.616023 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f21380-7d21-4149-ad50-2e98cf57faf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a235ce07c1264370bdfb7d16112c9c3b, user_id=d763750b633e4c72baf7f7fb42fa983d, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.618755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f21380-7d21-4149-ad50-2e98cf57faf1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.625776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9de3c452-da5d-454d-a245-be678c25083c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:04.627013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9de3c452-da5d-454d-a245-be678c25083c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.635307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 961/1938] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:04.644347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb569d5-cc8a-4e58-b5f2-7ddfde2122fd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.644711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb569d5-cc8a-4e58-b5f2-7ddfde2122fd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.644994 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb569d5-cc8a-4e58-b5f2-7ddfde2122fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.664250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8fb569d5-cc8a-4e58-b5f2-7ddfde2122fd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:04.665712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8fb569d5-cc8a-4e58-b5f2-7ddfde2122fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.674154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 962/1939] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:04.678346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb569d5-cc8a-4e58-b5f2-7ddfde2122fd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.688033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb569d5-cc8a-4e58-b5f2-7ddfde2122fd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.692715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e8b65-a4e0-4ed4-a074-7ac2a4253a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.693302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e8b65-a4e0-4ed4-a074-7ac2a4253a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.693584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e8b65-a4e0-4ed4-a074-7ac2a4253a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.693858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e8b65-a4e0-4ed4-a074-7ac2a4253a00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.694532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e8b65-a4e0-4ed4-a074-7ac2a4253a00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:04.694831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e8b65-a4e0-4ed4-a074-7ac2a4253a00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.696424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e8b65-a4e0-4ed4-a074-7ac2a4253a00 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.708890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 963/1940] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:26:04.713606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e8b65-a4e0-4ed4-a074-7ac2a4253a00 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.715897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 978/1941] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:04 2022] PUT /identity/v3/projects/a235ce07c1264370bdfb7d16112c9c3b/users/d763750b633e4c72baf7f7fb42fa983d/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:04.728815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff17b9-f925-40a5-ac23-fe7bb3412764 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.729219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff17b9-f925-40a5-ac23-fe7bb3412764 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.729493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e8b65-a4e0-4ed4-a074-7ac2a4253a00 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.729605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff17b9-f925-40a5-ac23-fe7bb3412764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.770283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-71ff17b9-f925-40a5-ac23-fe7bb3412764 None None] MFA Rules not processed for user `d763750b633e4c72baf7f7fb42fa983d`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:04.771730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-71ff17b9-f925-40a5-ac23-fe7bb3412764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.790742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e4cce-03a2-4aa9-a5f6-f64a06821ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.791360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e4cce-03a2-4aa9-a5f6-f64a06821ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.791640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e4cce-03a2-4aa9-a5f6-f64a06821ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.792237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e4cce-03a2-4aa9-a5f6-f64a06821ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.792573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e4cce-03a2-4aa9-a5f6-f64a06821ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.794490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e4cce-03a2-4aa9-a5f6-f64a06821ff2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.802989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:04.846779 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 979/1942] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:04.852448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff17b9-f925-40a5-ac23-fe7bb3412764 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.853789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-71ff17b9-f925-40a5-ac23-fe7bb3412764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:04.868314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff17b9-f925-40a5-ac23-fe7bb3412764 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:04.873438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de61c1-6f56-49da-90e0-ded3199aad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:04.874207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de61c1-6f56-49da-90e0-ded3199aad41 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:04.874561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de61c1-6f56-49da-90e0-ded3199aad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:04.874979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de61c1-6f56-49da-90e0-ded3199aad41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:04.875628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de61c1-6f56-49da-90e0-ded3199aad41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:04.877534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de61c1-6f56-49da-90e0-ded3199aad41 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:04.924279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 964/1943] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/projects => generated 399 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:04.928166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e4cce-03a2-4aa9-a5f6-f64a06821ff2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.977413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 980/1944] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:04.980377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de61c1-6f56-49da-90e0-ded3199aad41 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:04.994757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e4cce-03a2-4aa9-a5f6-f64a06821ff2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.056435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de61c1-6f56-49da-90e0-ded3199aad41 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.061485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d138e45-8731-4f3e-8f0c-a8941f3a39bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.062180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d138e45-8731-4f3e-8f0c-a8941f3a39bf admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.062476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d138e45-8731-4f3e-8f0c-a8941f3a39bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.063816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d138e45-8731-4f3e-8f0c-a8941f3a39bf admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.063816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d138e45-8731-4f3e-8f0c-a8941f3a39bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.065639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d138e45-8731-4f3e-8f0c-a8941f3a39bf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.070183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec759d54-497d-4aa8-8d01-a94d0344fb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.070791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec759d54-497d-4aa8-8d01-a94d0344fb3a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.071084 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec759d54-497d-4aa8-8d01-a94d0344fb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.071339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec759d54-497d-4aa8-8d01-a94d0344fb3a admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.071962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec759d54-497d-4aa8-8d01-a94d0344fb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.074012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec759d54-497d-4aa8-8d01-a94d0344fb3a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.181550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 965/1946] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/users => generated 300 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:05.181752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 981/1946] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:04 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:05.187883 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d138e45-8731-4f3e-8f0c-a8941f3a39bf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.189491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec759d54-497d-4aa8-8d01-a94d0344fb3a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.198263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d138e45-8731-4f3e-8f0c-a8941f3a39bf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.199543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec759d54-497d-4aa8-8d01-a94d0344fb3a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.204555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87828b60-fcc4-4e31-be20-a6993ded9084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.205181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87828b60-fcc4-4e31-be20-a6993ded9084 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.205484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87828b60-fcc4-4e31-be20-a6993ded9084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.205760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87828b60-fcc4-4e31-be20-a6993ded9084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.206521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87828b60-fcc4-4e31-be20-a6993ded9084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.207357 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5923b0-ce32-49a0-9daf-21d30866b747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.207962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5923b0-ce32-49a0-9daf-21d30866b747 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.208065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87828b60-fcc4-4e31-be20-a6993ded9084 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.208809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5923b0-ce32-49a0-9daf-21d30866b747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.209123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5923b0-ce32-49a0-9daf-21d30866b747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.209985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5923b0-ce32-49a0-9daf-21d30866b747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.218505 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5923b0-ce32-49a0-9daf-21d30866b747 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.222464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 982/1947] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:26:05.222601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 966/1948] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:26:05.226377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5923b0-ce32-49a0-9daf-21d30866b747 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.226509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87828b60-fcc4-4e31-be20-a6993ded9084 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.237092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5923b0-ce32-49a0-9daf-21d30866b747 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.237351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87828b60-fcc4-4e31-be20-a6993ded9084 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.242590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ae510-03bb-4a32-8a9f-1473455c96b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.243261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ae510-03bb-4a32-8a9f-1473455c96b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.243568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ae510-03bb-4a32-8a9f-1473455c96b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.243859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ead0b4-1b33-4f84-97f9-0b80dfdc4085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.243995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ae510-03bb-4a32-8a9f-1473455c96b3 admin admin] PATH_INFO: `/v3/projects/e93e7c25504d462da84bdfd5b8947426/users/c8c59d3aed7c4a26b8b12755a9560e9c/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.244601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ead0b4-1b33-4f84-97f9-0b80dfdc4085 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.244953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ead0b4-1b33-4f84-97f9-0b80dfdc4085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.245297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ead0b4-1b33-4f84-97f9-0b80dfdc4085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.246045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ead0b4-1b33-4f84-97f9-0b80dfdc4085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.247906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ead0b4-1b33-4f84-97f9-0b80dfdc4085 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.262156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ae510-03bb-4a32-8a9f-1473455c96b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93e7c25504d462da84bdfd5b8947426, user_id=c8c59d3aed7c4a26b8b12755a9560e9c, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.267078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 983/1949] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:05.267208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ae510-03bb-4a32-8a9f-1473455c96b3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.268879 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ead0b4-1b33-4f84-97f9-0b80dfdc4085 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.279208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ead0b4-1b33-4f84-97f9-0b80dfdc4085 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.284494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba3c1-aae2-4d8d-a028-ab30299d7d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.285152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba3c1-aae2-4d8d-a028-ab30299d7d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.285501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba3c1-aae2-4d8d-a028-ab30299d7d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.285776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba3c1-aae2-4d8d-a028-ab30299d7d97 admin admin] PATH_INFO: `/v3/projects/eaf297066c374f649c66692b9db9828b/users/584b8dc3c71049cd956e03333d8a344c/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.301221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ba3c1-aae2-4d8d-a028-ab30299d7d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf297066c374f649c66692b9db9828b, user_id=584b8dc3c71049cd956e03333d8a344c, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.304014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ba3c1-aae2-4d8d-a028-ab30299d7d97 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.397279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 967/1950] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:05 2022] PUT /identity/v3/projects/e93e7c25504d462da84bdfd5b8947426/users/c8c59d3aed7c4a26b8b12755a9560e9c/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:05.401211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ae510-03bb-4a32-8a9f-1473455c96b3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.432968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:05.442225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 984/1951] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:05 2022] PUT /identity/v3/projects/eaf297066c374f649c66692b9db9828b/users/584b8dc3c71049cd956e03333d8a344c/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:05.453814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa711d40-3ef6-4313-87f7-635fb1761c87 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.454272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa711d40-3ef6-4313-87f7-635fb1761c87 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.454550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa711d40-3ef6-4313-87f7-635fb1761c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.489651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ae510-03bb-4a32-8a9f-1473455c96b3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.492198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fa711d40-3ef6-4313-87f7-635fb1761c87 None None] MFA Rules not processed for user `584b8dc3c71049cd956e03333d8a344c`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:05.493451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fa711d40-3ef6-4313-87f7-635fb1761c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:05.531400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:05.558209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a178bca-07d3-4cd6-b855-cf357d677fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.558818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a178bca-07d3-4cd6-b855-cf357d677fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.559100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a178bca-07d3-4cd6-b855-cf357d677fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.559366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a178bca-07d3-4cd6-b855-cf357d677fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.560018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a178bca-07d3-4cd6-b855-cf357d677fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.562184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a178bca-07d3-4cd6-b855-cf357d677fa3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.572362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 968/1952] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:26:05.576456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a178bca-07d3-4cd6-b855-cf357d677fa3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.578613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 985/1953] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:05.583225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa711d40-3ef6-4313-87f7-635fb1761c87 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.588169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a178bca-07d3-4cd6-b855-cf357d677fa3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.594581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b05e20-6d04-43a7-8c03-196f4354635d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.595221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa711d40-3ef6-4313-87f7-635fb1761c87 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.595389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b05e20-6d04-43a7-8c03-196f4354635d admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.595863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b05e20-6d04-43a7-8c03-196f4354635d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.596209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b05e20-6d04-43a7-8c03-196f4354635d admin admin] PATH_INFO: `/v3/projects/e93e7c25504d462da84bdfd5b8947426/users/c8c59d3aed7c4a26b8b12755a9560e9c/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.598312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b05e20-6d04-43a7-8c03-196f4354635d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93e7c25504d462da84bdfd5b8947426, user_id=c8c59d3aed7c4a26b8b12755a9560e9c, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.600037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38790695-6a4e-4ad0-aaef-949e963dd128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.600801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38790695-6a4e-4ad0-aaef-949e963dd128 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.600966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b05e20-6d04-43a7-8c03-196f4354635d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.601215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38790695-6a4e-4ad0-aaef-949e963dd128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.601571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38790695-6a4e-4ad0-aaef-949e963dd128 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.602313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38790695-6a4e-4ad0-aaef-949e963dd128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.604470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38790695-6a4e-4ad0-aaef-949e963dd128 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.615634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 969/1954] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:05 2022] PUT /identity/v3/projects/e93e7c25504d462da84bdfd5b8947426/users/c8c59d3aed7c4a26b8b12755a9560e9c/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:05.627293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10c714-0163-47e7-a6b6-38050b7c06fe None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.627738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10c714-0163-47e7-a6b6-38050b7c06fe None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.628153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10c714-0163-47e7-a6b6-38050b7c06fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.656662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-dc10c714-0163-47e7-a6b6-38050b7c06fe None None] MFA Rules not processed for user `c8c59d3aed7c4a26b8b12755a9560e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:05.657900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dc10c714-0163-47e7-a6b6-38050b7c06fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:05.698727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 986/1955] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:05 2022] POST /identity/v3/projects => generated 399 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:05.702795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38790695-6a4e-4ad0-aaef-949e963dd128 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.739167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 970/1956] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:05.752139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10c714-0163-47e7-a6b6-38050b7c06fe None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.753630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dc10c714-0163-47e7-a6b6-38050b7c06fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:05.780681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38790695-6a4e-4ad0-aaef-949e963dd128 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.785827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78f307-bc44-4431-992e-ed2760546f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.786619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78f307-bc44-4431-992e-ed2760546f33 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.786905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78f307-bc44-4431-992e-ed2760546f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.787179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78f307-bc44-4431-992e-ed2760546f33 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.787828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78f307-bc44-4431-992e-ed2760546f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.790395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78f307-bc44-4431-992e-ed2760546f33 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.815959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10c714-0163-47e7-a6b6-38050b7c06fe None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.866160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efad60a-f47c-4540-b74f-5aa6bba775fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.866864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efad60a-f47c-4540-b74f-5aa6bba775fa service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.867151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efad60a-f47c-4540-b74f-5aa6bba775fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.867428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efad60a-f47c-4540-b74f-5aa6bba775fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.914553 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 987/1957] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:05 2022] POST /identity/v3/users => generated 306 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:05.915953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78f307-bc44-4431-992e-ed2760546f33 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.926669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78f307-bc44-4431-992e-ed2760546f33 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.930434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efad60a-f47c-4540-b74f-5aa6bba775fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.931629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db8b49-4638-46b7-9361-62e58ced8c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.932068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efad60a-f47c-4540-b74f-5aa6bba775fa service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.932325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db8b49-4638-46b7-9361-62e58ced8c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.932832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db8b49-4638-46b7-9361-62e58ced8c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.933290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db8b49-4638-46b7-9361-62e58ced8c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.933984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db8b49-4638-46b7-9361-62e58ced8c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.936993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db8b49-4638-46b7-9361-62e58ced8c55 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:05.943281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 971/1958] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:05.948477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 988/1959] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:26:05.952776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efad60a-f47c-4540-b74f-5aa6bba775fa service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:05.964721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efad60a-f47c-4540-b74f-5aa6bba775fa service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:05.970457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3467a6-4b4d-4c47-9e5e-fecbc5ad0e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:05.971295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3467a6-4b4d-4c47-9e5e-fecbc5ad0e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:05.972835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3467a6-4b4d-4c47-9e5e-fecbc5ad0e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:05.972835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3467a6-4b4d-4c47-9e5e-fecbc5ad0e75 admin admin] PATH_INFO: `/v3/projects/2d52240d3020417186db77ade5978f12/users/f939cc044bbc4165bb2fbd9779e2d8b7/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:05.989614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3467a6-4b4d-4c47-9e5e-fecbc5ad0e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d52240d3020417186db77ade5978f12, user_id=f939cc044bbc4165bb2fbd9779e2d8b7, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:05.992264 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3467a6-4b4d-4c47-9e5e-fecbc5ad0e75 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:06.028753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:06.105329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 972/1960] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:05 2022] PUT /identity/v3/projects/2d52240d3020417186db77ade5978f12/users/f939cc044bbc4165bb2fbd9779e2d8b7/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:06.111204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db8b49-4638-46b7-9361-62e58ced8c55 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:06.121544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db8b49-4638-46b7-9361-62e58ced8c55 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:06.186767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c2ac8-0d1e-4755-bcfb-5ed3d850721f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:06.187372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c2ac8-0d1e-4755-bcfb-5ed3d850721f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:06.187652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c2ac8-0d1e-4755-bcfb-5ed3d850721f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:06.187921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c2ac8-0d1e-4755-bcfb-5ed3d850721f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:06.188570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c2ac8-0d1e-4755-bcfb-5ed3d850721f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:06.190253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c2ac8-0d1e-4755-bcfb-5ed3d850721f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:06.200573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 989/1961] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:06.205135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3467a6-4b4d-4c47-9e5e-fecbc5ad0e75 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:06.215283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3467a6-4b4d-4c47-9e5e-fecbc5ad0e75 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:06.220464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db8d9f-8c80-4aba-9d47-02fc2468f1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:06.221112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db8d9f-8c80-4aba-9d47-02fc2468f1c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:06.221379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db8d9f-8c80-4aba-9d47-02fc2468f1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:06.221646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db8d9f-8c80-4aba-9d47-02fc2468f1c1 admin admin] PATH_INFO: `/v3/projects/2d52240d3020417186db77ade5978f12/users/f939cc044bbc4165bb2fbd9779e2d8b7/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:06.223521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db8d9f-8c80-4aba-9d47-02fc2468f1c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d52240d3020417186db77ade5978f12, user_id=f939cc044bbc4165bb2fbd9779e2d8b7, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:06.226155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db8d9f-8c80-4aba-9d47-02fc2468f1c1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:06.236998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 973/1962] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:06 2022] PUT /identity/v3/projects/2d52240d3020417186db77ade5978f12/users/f939cc044bbc4165bb2fbd9779e2d8b7/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:06.248782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a6a8b-5967-4eca-a96e-7c25280bb543 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:06.248998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a6a8b-5967-4eca-a96e-7c25280bb543 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:06.249278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a6a8b-5967-4eca-a96e-7c25280bb543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:06.277643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ec1a6a8b-5967-4eca-a96e-7c25280bb543 None None] MFA Rules not processed for user `f939cc044bbc4165bb2fbd9779e2d8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:06.278887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ec1a6a8b-5967-4eca-a96e-7c25280bb543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:06.291530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:06.349424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 990/1963] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:06 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:06.714855 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:06.827693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db8d9f-8c80-4aba-9d47-02fc2468f1c1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:06.894877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db8d9f-8c80-4aba-9d47-02fc2468f1c1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:06.898406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86837c43-d99e-414b-854b-2e3b31f6097f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:06.899013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86837c43-d99e-414b-854b-2e3b31f6097f service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:06.899291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86837c43-d99e-414b-854b-2e3b31f6097f service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:06.899565 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86837c43-d99e-414b-854b-2e3b31f6097f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:06.908582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86837c43-d99e-414b-854b-2e3b31f6097f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:06.910059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86837c43-d99e-414b-854b-2e3b31f6097f service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:06.919713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 974/1964] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:06 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:07.380484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a6a8b-5967-4eca-a96e-7c25280bb543 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:07.390240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a6a8b-5967-4eca-a96e-7c25280bb543 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:07.393816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf342f-7fa6-42cb-afd1-f437b52edec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:07.394450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf342f-7fa6-42cb-afd1-f437b52edec1 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:07.394848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf342f-7fa6-42cb-afd1-f437b52edec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:07.394996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf342f-7fa6-42cb-afd1-f437b52edec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:07.457969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf342f-7fa6-42cb-afd1-f437b52edec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:07.459354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf342f-7fa6-42cb-afd1-f437b52edec1 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:07.468275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 991/1965] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:08.079117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86837c43-d99e-414b-854b-2e3b31f6097f service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:08.080634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-86837c43-d99e-414b-854b-2e3b31f6097f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:08.093386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86837c43-d99e-414b-854b-2e3b31f6097f service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:08.097618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb99f8e-f95c-4875-b4fc-70f61b71ce80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:08.098628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb99f8e-f95c-4875-b4fc-70f61b71ce80 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:08.098738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb99f8e-f95c-4875-b4fc-70f61b71ce80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:08.098945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb99f8e-f95c-4875-b4fc-70f61b71ce80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:08.163552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb99f8e-f95c-4875-b4fc-70f61b71ce80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:08.165644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb99f8e-f95c-4875-b4fc-70f61b71ce80 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:08.174830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 975/1966] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:17.464195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=51 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:18.170615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:36.086344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf342f-7fa6-42cb-afd1-f437b52edec1 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:36.154141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf342f-7fa6-42cb-afd1-f437b52edec1 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:36.159885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:36.160591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:36.160870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:36.161140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] PATH_INFO: `/v3/users/d763750b633e4c72baf7f7fb42fa983d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:36.162323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d763750b633e4c72baf7f7fb42fa983d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:36.164590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:36.442247 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd763750b633e4c72baf7f7fb42fa983d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:36.453311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd763750b633e4c72baf7f7fb42fa983d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:36.453311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd763750b633e4c72baf7f7fb42fa983d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:36.456126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd763750b633e4c72baf7f7fb42fa983d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:36.473549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd763750b633e4c72baf7f7fb42fa983d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:36.578950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 992/1967] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:36 2022] DELETE /identity/v3/users/d763750b633e4c72baf7f7fb42fa983d => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:36.580662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e0dee-261b-41a5-9a3c-06a3073f26a8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:36.581128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e0dee-261b-41a5-9a3c-06a3073f26a8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:36.581352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e0dee-261b-41a5-9a3c-06a3073f26a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:36.601299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-954e0dee-261b-41a5-9a3c-06a3073f26a8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:36.602563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-954e0dee-261b-41a5-9a3c-06a3073f26a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:36.613627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:36.615542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:36.697432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 976/1968] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:36.708635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c0e47-b55f-4f0a-a5a8-d72845ffcc4b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:36.708635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:36.708635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:36.708635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:36.708963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] PATH_INFO: `/v3/users/584b8dc3c71049cd956e03333d8a344c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:36.709705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=584b8dc3c71049cd956e03333d8a344c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:36.711712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:36.717906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e0dee-261b-41a5-9a3c-06a3073f26a8 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:36.720479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-954e0dee-261b-41a5-9a3c-06a3073f26a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:36.741310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:36.783198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e0dee-261b-41a5-9a3c-06a3073f26a8 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:36.786777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e47ae4-acb2-4cb4-9ce1-730f976ceae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:36.787407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e47ae4-acb2-4cb4-9ce1-730f976ceae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:36.787945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e47ae4-acb2-4cb4-9ce1-730f976ceae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:36.788217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e47ae4-acb2-4cb4-9ce1-730f976ceae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:36.798126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e47ae4-acb2-4cb4-9ce1-730f976ceae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:36.799624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e47ae4-acb2-4cb4-9ce1-730f976ceae2 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:36.810570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 977/1969] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:36.948252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '584b8dc3c71049cd956e03333d8a344c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:36.948835 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '584b8dc3c71049cd956e03333d8a344c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:36.949448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '584b8dc3c71049cd956e03333d8a344c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:36.956458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '584b8dc3c71049cd956e03333d8a344c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:36.968732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '584b8dc3c71049cd956e03333d8a344c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.073026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 993/1970] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:36 2022] DELETE /identity/v3/users/584b8dc3c71049cd956e03333d8a344c => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:37.086026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d594a72-78bf-446b-a464-3f8805046562 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:37.086528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d594a72-78bf-446b-a464-3f8805046562 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:37.086838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d594a72-78bf-446b-a464-3f8805046562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:37.114456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9d594a72-78bf-446b-a464-3f8805046562 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:37.114456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9d594a72-78bf-446b-a464-3f8805046562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:37.196170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 978/1971] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:37.206813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:37.209626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:37.265659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2174c1-8206-46e6-9822-1d5ae557a19f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:37.269248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da8a62-8cb0-4bb5-8653-2ebdb67c16e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:37.269852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da8a62-8cb0-4bb5-8653-2ebdb67c16e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:37.270160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da8a62-8cb0-4bb5-8653-2ebdb67c16e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:37.270431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da8a62-8cb0-4bb5-8653-2ebdb67c16e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:37.280105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da8a62-8cb0-4bb5-8653-2ebdb67c16e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:37.281564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da8a62-8cb0-4bb5-8653-2ebdb67c16e0 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:37.292824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 994/1972] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:37.295644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d594a72-78bf-446b-a464-3f8805046562 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:37.295644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9d594a72-78bf-446b-a464-3f8805046562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:37.307477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d594a72-78bf-446b-a464-3f8805046562 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:37.312151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:37.312788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:37.313103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:37.313384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] PATH_INFO: `/v3/projects/a235ce07c1264370bdfb7d16112c9c3b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:37.314698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a235ce07c1264370bdfb7d16112c9c3b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:37.316790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:37.463815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:37.473547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a235ce07c1264370bdfb7d16112c9c3b', 'request_id': 'req-49b28c84-039d-4591-9520-e60621b796e6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.479408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a235ce07c1264370bdfb7d16112c9c3b', 'request_id': 'req-49b28c84-039d-4591-9520-e60621b796e6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.486397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a235ce07c1264370bdfb7d16112c9c3b', 'request_id': 'req-49b28c84-039d-4591-9520-e60621b796e6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.600645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] The token cache is being invalidate because project a235ce07c1264370bdfb7d16112c9c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:37.601253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.601769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.603370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 979/1973] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:37 2022] DELETE /identity/v3/projects/a235ce07c1264370bdfb7d16112c9c3b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:37.607556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da8a62-8cb0-4bb5-8653-2ebdb67c16e0 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:37.617687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da8a62-8cb0-4bb5-8653-2ebdb67c16e0 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:37.688871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:37.689515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:37.689798 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:37.690111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] PATH_INFO: `/v3/users/c8c59d3aed7c4a26b8b12755a9560e9c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:37.691253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c59d3aed7c4a26b8b12755a9560e9c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:37.693213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:37.714939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:37.716711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:37.734083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b28c84-039d-4591-9520-e60621b796e6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:37.754163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:37.755012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:37.755444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:37.755865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] PATH_INFO: `/v3/projects/eaf297066c374f649c66692b9db9828b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:37.757670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf297066c374f649c66692b9db9828b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:37.760107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:37.934293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf297066c374f649c66692b9db9828b', 'request_id': 'req-5caf5324-fb4a-4d4a-9710-e581c932f177'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.953925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf297066c374f649c66692b9db9828b', 'request_id': 'req-5caf5324-fb4a-4d4a-9710-e581c932f177'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.962134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf297066c374f649c66692b9db9828b', 'request_id': 'req-5caf5324-fb4a-4d4a-9710-e581c932f177'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.964717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:37.969592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c59d3aed7c4a26b8b12755a9560e9c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.970738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c59d3aed7c4a26b8b12755a9560e9c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.971756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c59d3aed7c4a26b8b12755a9560e9c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.980559 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c59d3aed7c4a26b8b12755a9560e9c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:37.994573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c59d3aed7c4a26b8b12755a9560e9c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.028863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] The token cache is being invalidate because project eaf297066c374f649c66692b9db9828b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:38.029151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.029949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.031469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 980/1974] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:37 2022] DELETE /identity/v3/projects/eaf297066c374f649c66692b9db9828b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:38.035220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:38.036507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:38.108745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf5324-fb4a-4d4a-9710-e581c932f177 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:38.113657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:38.114359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:38.114651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:38.114924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] PATH_INFO: `/v3/users/f939cc044bbc4165bb2fbd9779e2d8b7` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:38.116093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f939cc044bbc4165bb2fbd9779e2d8b7)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:38.118276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:38.156776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 995/1975] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:37 2022] DELETE /identity/v3/users/c8c59d3aed7c4a26b8b12755a9560e9c => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:38.334322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f939cc044bbc4165bb2fbd9779e2d8b7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.335156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f939cc044bbc4165bb2fbd9779e2d8b7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.335942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f939cc044bbc4165bb2fbd9779e2d8b7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.340479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:38.435106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f939cc044bbc4165bb2fbd9779e2d8b7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.443802 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f939cc044bbc4165bb2fbd9779e2d8b7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.457179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:38.458773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:38.470170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 981/1976] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:38 2022] DELETE /identity/v3/users/f939cc044bbc4165bb2fbd9779e2d8b7 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:38.537345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203eb1-4563-4881-ad0e-34cbbd11e9df admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:38.598572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:38.599263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:38.599541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:38.599820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] PATH_INFO: `/v3/users/1f35fad1139349ca94e6b7356c01f099` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:38.600960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f35fad1139349ca94e6b7356c01f099)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:38.602948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:38.666335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:38.669310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:38.703906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38214c1d-0f16-4e58-bd8b-9c097f191298 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:38.709925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:38.710675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:38.711106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:38.711553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] PATH_INFO: `/v3/projects/e93e7c25504d462da84bdfd5b8947426` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:38.713014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93e7c25504d462da84bdfd5b8947426)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:38.715631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:38.882698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f35fad1139349ca94e6b7356c01f099'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.883495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f35fad1139349ca94e6b7356c01f099'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.884476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f35fad1139349ca94e6b7356c01f099'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.892695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f35fad1139349ca94e6b7356c01f099'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.903436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:38.905392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-658a9a47-e905-4408-80db-3cf70b3c953c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f35fad1139349ca94e6b7356c01f099'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.945201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93e7c25504d462da84bdfd5b8947426', 'request_id': 'req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.951339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93e7c25504d462da84bdfd5b8947426', 'request_id': 'req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:38.957957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93e7c25504d462da84bdfd5b8947426', 'request_id': 'req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:39.046930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 996/1977] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:38 2022] DELETE /identity/v3/users/1f35fad1139349ca94e6b7356c01f099 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:39.051276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58375a-f91b-4539-a6fd-bd4d3b7590ed None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.051562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58375a-f91b-4539-a6fd-bd4d3b7590ed None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.051836 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58375a-f91b-4539-a6fd-bd4d3b7590ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.070680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ae58375a-f91b-4539-a6fd-bd4d3b7590ed None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:39.071825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ae58375a-f91b-4539-a6fd-bd4d3b7590ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:39.125754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] The token cache is being invalidate because project e93e7c25504d462da84bdfd5b8947426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:39.133021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:39.133021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:39.133021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 982/1978] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:38 2022] DELETE /identity/v3/projects/e93e7c25504d462da84bdfd5b8947426 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:39.133021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.133021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:39.133938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 997/1979] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:39.135280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf49a5-86c7-4f30-8c72-dabe4db2a01d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.135626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf49a5-86c7-4f30-8c72-dabe4db2a01d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.135906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf49a5-86c7-4f30-8c72-dabe4db2a01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.156920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-87cf49a5-86c7-4f30-8c72-dabe4db2a01d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:39.158180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-87cf49a5-86c7-4f30-8c72-dabe4db2a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:39.168227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 998/1980] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:39.168920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf49a5-86c7-4f30-8c72-dabe4db2a01d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.170984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-87cf49a5-86c7-4f30-8c72-dabe4db2a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:39.206263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c87c5-c2f6-4c9b-b621-a9fc88e336c8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.210895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.211556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.211830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.212100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] PATH_INFO: `/v3/projects/2d52240d3020417186db77ade5978f12` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.213235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d52240d3020417186db77ade5978f12)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.215252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.228285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf49a5-86c7-4f30-8c72-dabe4db2a01d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.232998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13906aa-16ec-4282-9a15-a0113dc4e393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.233630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13906aa-16ec-4282-9a15-a0113dc4e393 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.233924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13906aa-16ec-4282-9a15-a0113dc4e393 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.234244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13906aa-16ec-4282-9a15-a0113dc4e393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.245395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13906aa-16ec-4282-9a15-a0113dc4e393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.247157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13906aa-16ec-4282-9a15-a0113dc4e393 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.257807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 999/1981] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:39.259289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fc784-b466-4d8f-842b-2e2854d6cc11 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.259691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fc784-b466-4d8f-842b-2e2854d6cc11 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.259979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fc784-b466-4d8f-842b-2e2854d6cc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.298982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a71fc784-b466-4d8f-842b-2e2854d6cc11 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:39.306312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a71fc784-b466-4d8f-842b-2e2854d6cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:39.308680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1000/1982] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:39.319408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fc784-b466-4d8f-842b-2e2854d6cc11 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.331070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fc784-b466-4d8f-842b-2e2854d6cc11 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.335465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf3a90-afc7-4bf2-a249-4fe87891d5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.336076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf3a90-afc7-4bf2-a249-4fe87891d5cd admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.336361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf3a90-afc7-4bf2-a249-4fe87891d5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.336635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf3a90-afc7-4bf2-a249-4fe87891d5cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.337293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf3a90-afc7-4bf2-a249-4fe87891d5cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:39.337596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf3a90-afc7-4bf2-a249-4fe87891d5cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.339391 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf3a90-afc7-4bf2-a249-4fe87891d5cd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.353463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1001/1983] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:26:39.358151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf3a90-afc7-4bf2-a249-4fe87891d5cd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.368581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf3a90-afc7-4bf2-a249-4fe87891d5cd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.368805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d52240d3020417186db77ade5978f12', 'request_id': 'req-4152b334-90a1-4099-aed7-104a1ec35afc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:39.376534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:39.376534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d52240d3020417186db77ade5978f12', 'request_id': 'req-4152b334-90a1-4099-aed7-104a1ec35afc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:39.382789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d52240d3020417186db77ade5978f12', 'request_id': 'req-4152b334-90a1-4099-aed7-104a1ec35afc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:39.429813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c113bef-689f-4403-bfac-f14cf42555b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.430451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c113bef-689f-4403-bfac-f14cf42555b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.430733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c113bef-689f-4403-bfac-f14cf42555b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.431007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c113bef-689f-4403-bfac-f14cf42555b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.431639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c113bef-689f-4403-bfac-f14cf42555b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.433570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c113bef-689f-4403-bfac-f14cf42555b9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.448872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:39.451453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] The token cache is being invalidate because project 2d52240d3020417186db77ade5978f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:39.457836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:39.457836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4152b334-90a1-4099-aed7-104a1ec35afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:39.457836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 983/1984] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:39 2022] DELETE /identity/v3/projects/2d52240d3020417186db77ade5978f12 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:39.458824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361d5a-f150-49d6-9604-1d6e422df4a3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.459109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361d5a-f150-49d6-9604-1d6e422df4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.459410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361d5a-f150-49d6-9604-1d6e422df4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.481913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d0361d5a-f150-49d6-9604-1d6e422df4a3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:39.483451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d0361d5a-f150-49d6-9604-1d6e422df4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:39.492792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 984/1985] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:39.502157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d6b09-b6cb-4936-b97f-30c96fc6d039 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.502446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d6b09-b6cb-4936-b97f-30c96fc6d039 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.502756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d6b09-b6cb-4936-b97f-30c96fc6d039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.524281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0a7d6b09-b6cb-4936-b97f-30c96fc6d039 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:39.525430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0a7d6b09-b6cb-4936-b97f-30c96fc6d039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:39.535719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1002/1986] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:39 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:39.539779 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c113bef-689f-4403-bfac-f14cf42555b9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.541666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9c113bef-689f-4403-bfac-f14cf42555b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:39.545128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 985/1987] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:39.547766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d6b09-b6cb-4936-b97f-30c96fc6d039 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.559116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d6b09-b6cb-4936-b97f-30c96fc6d039 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.615687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c113bef-689f-4403-bfac-f14cf42555b9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.621692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe99587-e348-4e4d-87b8-3a7f38827a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.622545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe99587-e348-4e4d-87b8-3a7f38827a13 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.622852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe99587-e348-4e4d-87b8-3a7f38827a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.623121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe99587-e348-4e4d-87b8-3a7f38827a13 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.623754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe99587-e348-4e4d-87b8-3a7f38827a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.625712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe99587-e348-4e4d-87b8-3a7f38827a13 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.631979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260833a-d31c-45bd-8efe-7d538bc1f624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.632701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260833a-d31c-45bd-8efe-7d538bc1f624 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.632876 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260833a-d31c-45bd-8efe-7d538bc1f624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.633247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260833a-d31c-45bd-8efe-7d538bc1f624 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.633808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260833a-d31c-45bd-8efe-7d538bc1f624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:39.634155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260833a-d31c-45bd-8efe-7d538bc1f624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.635806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260833a-d31c-45bd-8efe-7d538bc1f624 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.648463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 986/1988] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:26:39.653048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260833a-d31c-45bd-8efe-7d538bc1f624 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.663416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260833a-d31c-45bd-8efe-7d538bc1f624 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.668191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebf0f8-12a9-4646-9b85-8d1998b35cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.668755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf0f8-12a9-4646-9b85-8d1998b35cfe admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.669034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf0f8-12a9-4646-9b85-8d1998b35cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.669307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf0f8-12a9-4646-9b85-8d1998b35cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.669983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebf0f8-12a9-4646-9b85-8d1998b35cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.671878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebf0f8-12a9-4646-9b85-8d1998b35cfe admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.784978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1003/1989] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:39 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:39.786856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe99587-e348-4e4d-87b8-3a7f38827a13 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.789851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 987/1990] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:39 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:39.790681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebf0f8-12a9-4646-9b85-8d1998b35cfe admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.798026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cfe99587-e348-4e4d-87b8-3a7f38827a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:39.814728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe99587-e348-4e4d-87b8-3a7f38827a13 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.864084 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebf0f8-12a9-4646-9b85-8d1998b35cfe admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.868977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcad03-2c77-4bf5-8732-aadeb69cb221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.869519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcad03-2c77-4bf5-8732-aadeb69cb221 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.869802 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcad03-2c77-4bf5-8732-aadeb69cb221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.870110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcad03-2c77-4bf5-8732-aadeb69cb221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.870773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcad03-2c77-4bf5-8732-aadeb69cb221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.873076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcad03-2c77-4bf5-8732-aadeb69cb221 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.873675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.874547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.875022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.876361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] PATH_INFO: `/v3/projects/7f363d569e794bdc94d4ec77f637dd5e` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.877675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f363d569e794bdc94d4ec77f637dd5e)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.880197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.883722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 988/1991] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:39.884647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcad03-2c77-4bf5-8732-aadeb69cb221 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:39.897980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcad03-2c77-4bf5-8732-aadeb69cb221 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:39.906600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13c563-3248-40d9-958f-153763a53604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:39.907223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13c563-3248-40d9-958f-153763a53604 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:39.908036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13c563-3248-40d9-958f-153763a53604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:39.908949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13c563-3248-40d9-958f-153763a53604 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:39.910018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13c563-3248-40d9-958f-153763a53604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:39.912856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13c563-3248-40d9-958f-153763a53604 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:39.999637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f363d569e794bdc94d4ec77f637dd5e', 'request_id': 'req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.005282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f363d569e794bdc94d4ec77f637dd5e', 'request_id': 'req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.012050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 989/1992] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:39 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:40.012890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13c563-3248-40d9-958f-153763a53604 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.022821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13c563-3248-40d9-958f-153763a53604 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.040465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:40.079899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a3fc9-f493-4a22-b968-0d2d7bfcb938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.080506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a3fc9-f493-4a22-b968-0d2d7bfcb938 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.080790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a3fc9-f493-4a22-b968-0d2d7bfcb938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.081068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a3fc9-f493-4a22-b968-0d2d7bfcb938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.081716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a3fc9-f493-4a22-b968-0d2d7bfcb938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.083386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a3fc9-f493-4a22-b968-0d2d7bfcb938 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:40.094801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 990/1993] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:40.095780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a3fc9-f493-4a22-b968-0d2d7bfcb938 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.105376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a3fc9-f493-4a22-b968-0d2d7bfcb938 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.110600 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4692a8-7931-498d-8273-efa2fdd091e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.113340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4692a8-7931-498d-8273-efa2fdd091e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.113340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4692a8-7931-498d-8273-efa2fdd091e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.113340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4692a8-7931-498d-8273-efa2fdd091e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.113340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4692a8-7931-498d-8273-efa2fdd091e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.114071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4692a8-7931-498d-8273-efa2fdd091e1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:40.125233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 991/1994] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:40.126119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4692a8-7931-498d-8273-efa2fdd091e1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.131062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f363d569e794bdc94d4ec77f637dd5e', 'request_id': 'req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.135911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4692a8-7931-498d-8273-efa2fdd091e1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.141189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea671ab-baea-4523-9bfd-9ab0903cce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.148112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea671ab-baea-4523-9bfd-9ab0903cce1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.148112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea671ab-baea-4523-9bfd-9ab0903cce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.148112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea671ab-baea-4523-9bfd-9ab0903cce1e admin admin] PATH_INFO: `/v3/projects/9e9e749165c544e7b311a4f350edf706/users/e652482a084246e98231bb97bae4064c/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.148371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] The token cache is being invalidate because project 7f363d569e794bdc94d4ec77f637dd5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:40.148371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.148371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.148371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1004/1995] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:39 2022] DELETE /identity/v3/projects/7f363d569e794bdc94d4ec77f637dd5e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:40.148371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.149285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:40.157173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea671ab-baea-4523-9bfd-9ab0903cce1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9e749165c544e7b311a4f350edf706, user_id=e652482a084246e98231bb97bae4064c, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.159952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea671ab-baea-4523-9bfd-9ab0903cce1e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:40.162729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6de66-8f9f-45e7-80b7-6d0790a39dc1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.167675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa9df4-c193-4bb5-ade0-4288c503146d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.168315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa9df4-c193-4bb5-ade0-4288c503146d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.168597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa9df4-c193-4bb5-ade0-4288c503146d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.168873 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa9df4-c193-4bb5-ade0-4288c503146d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.170253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa9df4-c193-4bb5-ade0-4288c503146d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.171864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa9df4-c193-4bb5-ade0-4288c503146d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:40.174207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:40.182944 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1005/1996] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:40.183894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa9df4-c193-4bb5-ade0-4288c503146d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.186166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-25aa9df4-c193-4bb5-ade0-4288c503146d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:40.198711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa9df4-c193-4bb5-ade0-4288c503146d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.203484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.204143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.204430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.204708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] PATH_INFO: `/v3/users/52d974fe2d8f46c0acaac2490ab833cf` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.205847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d974fe2d8f46c0acaac2490ab833cf)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.207834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:40.222933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 992/1997] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:40 2022] PUT /identity/v3/projects/9e9e749165c544e7b311a4f350edf706/users/e652482a084246e98231bb97bae4064c/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:40.223815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea671ab-baea-4523-9bfd-9ab0903cce1e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.285966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea671ab-baea-4523-9bfd-9ab0903cce1e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.292066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f494f-fc24-4eb0-9f98-acd99dbfe938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.292780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f494f-fc24-4eb0-9f98-acd99dbfe938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.293061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f494f-fc24-4eb0-9f98-acd99dbfe938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.293391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f494f-fc24-4eb0-9f98-acd99dbfe938 admin admin] PATH_INFO: `/v3/projects/24305a12daa54c6c94ec0fdd234ab400/users/70e1b5a6031a4de4b53c043f21ca0738/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.311733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f494f-fc24-4eb0-9f98-acd99dbfe938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24305a12daa54c6c94ec0fdd234ab400, user_id=70e1b5a6031a4de4b53c043f21ca0738, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.314544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f494f-fc24-4eb0-9f98-acd99dbfe938 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:40.434889 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 993/1998] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:40 2022] PUT /identity/v3/projects/24305a12daa54c6c94ec0fdd234ab400/users/70e1b5a6031a4de4b53c043f21ca0738/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:40.436667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41685b78-33f7-4f3d-8c91-273c87a5c4ce None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.436954 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41685b78-33f7-4f3d-8c91-273c87a5c4ce None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.437229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41685b78-33f7-4f3d-8c91-273c87a5c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.464367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-41685b78-33f7-4f3d-8c91-273c87a5c4ce None None] MFA Rules not processed for user `e652482a084246e98231bb97bae4064c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:40.465581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-41685b78-33f7-4f3d-8c91-273c87a5c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:40.484111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d974fe2d8f46c0acaac2490ab833cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.484699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d974fe2d8f46c0acaac2490ab833cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.485311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d974fe2d8f46c0acaac2490ab833cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.492369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d974fe2d8f46c0acaac2490ab833cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.504411 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d974fe2d8f46c0acaac2490ab833cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:40.522305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:40.532811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 994/1999] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:40.534120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039b1b-9a1f-4bba-ae39-44c0d6ae45a2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.534404 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039b1b-9a1f-4bba-ae39-44c0d6ae45a2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.534676 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039b1b-9a1f-4bba-ae39-44c0d6ae45a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.563270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-db039b1b-9a1f-4bba-ae39-44c0d6ae45a2 None None] MFA Rules not processed for user `70e1b5a6031a4de4b53c043f21ca0738`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:40.564519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-db039b1b-9a1f-4bba-ae39-44c0d6ae45a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:40.599914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1006/2000] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:40 2022] DELETE /identity/v3/users/52d974fe2d8f46c0acaac2490ab833cf => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:40.600826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.602234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:40.632999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 995/2001] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:40.637648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db039b1b-9a1f-4bba-ae39-44c0d6ae45a2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.639743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-db039b1b-9a1f-4bba-ae39-44c0d6ae45a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:40.674903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a81b4-2ae0-4ccd-814d-717e5ccaabde admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.678859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff4ae6-1e0f-44ac-a073-fcc10e711f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.679496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff4ae6-1e0f-44ac-a073-fcc10e711f4a service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.679775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff4ae6-1e0f-44ac-a073-fcc10e711f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.680045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff4ae6-1e0f-44ac-a073-fcc10e711f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.716444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db039b1b-9a1f-4bba-ae39-44c0d6ae45a2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.741178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff4ae6-1e0f-44ac-a073-fcc10e711f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.742647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff4ae6-1e0f-44ac-a073-fcc10e711f4a service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:40.752305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1007/2002] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:40.774086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e12d40-b584-4b94-9037-b8e80d19fa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.774691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e12d40-b584-4b94-9037-b8e80d19fa79 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.774977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e12d40-b584-4b94-9037-b8e80d19fa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.775278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e12d40-b584-4b94-9037-b8e80d19fa79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.775895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e12d40-b584-4b94-9037-b8e80d19fa79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.777782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e12d40-b584-4b94-9037-b8e80d19fa79 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:40.892257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 996/2003] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:40 2022] POST /identity/v3/projects => generated 385 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:40.896484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff4ae6-1e0f-44ac-a073-fcc10e711f4a service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:40.908066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff4ae6-1e0f-44ac-a073-fcc10e711f4a service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:40.929570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:40.973643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bfa11-484c-42ae-beac-dd8d028ada67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:40.974220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bfa11-484c-42ae-beac-dd8d028ada67 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:40.974500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bfa11-484c-42ae-beac-dd8d028ada67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:40.974774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bfa11-484c-42ae-beac-dd8d028ada67 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:40.975445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bfa11-484c-42ae-beac-dd8d028ada67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:40.977421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bfa11-484c-42ae-beac-dd8d028ada67 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.077070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1008/2004] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:40 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:41.077217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e12d40-b584-4b94-9037-b8e80d19fa79 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.077692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-89e12d40-b584-4b94-9037-b8e80d19fa79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:41.078433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bfa11-484c-42ae-beac-dd8d028ada67 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.089311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bfa11-484c-42ae-beac-dd8d028ada67 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.093113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e12d40-b584-4b94-9037-b8e80d19fa79 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.095946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9497d-5783-4d9a-b761-642c70122067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.096526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9497d-5783-4d9a-b761-642c70122067 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.096717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9497d-5783-4d9a-b761-642c70122067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.097057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9497d-5783-4d9a-b761-642c70122067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.102016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9497d-5783-4d9a-b761-642c70122067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.102136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.102254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9497d-5783-4d9a-b761-642c70122067 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.102330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.102330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.102330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] PATH_INFO: `/v3/projects/c5ec4c189693458ab07bde1354c2a439` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.102330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ec4c189693458ab07bde1354c2a439)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.104070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.116455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1009/2005] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:26:41.120465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:41.120916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9497d-5783-4d9a-b761-642c70122067 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.131126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9497d-5783-4d9a-b761-642c70122067 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.135818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b169ef-d76c-4f72-a72c-d8cfd39814d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.136468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b169ef-d76c-4f72-a72c-d8cfd39814d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.136748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b169ef-d76c-4f72-a72c-d8cfd39814d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.137022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b169ef-d76c-4f72-a72c-d8cfd39814d7 admin admin] PATH_INFO: `/v3/projects/67b5ba4ab2034d58905342d90f663692/users/f0967244091c47f1991f90d4548aedd9/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.155392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b169ef-d76c-4f72-a72c-d8cfd39814d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b5ba4ab2034d58905342d90f663692, user_id=f0967244091c47f1991f90d4548aedd9, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.157595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b169ef-d76c-4f72-a72c-d8cfd39814d7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.260749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1010/2006] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:41 2022] PUT /identity/v3/projects/67b5ba4ab2034d58905342d90f663692/users/f0967244091c47f1991f90d4548aedd9/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:41.265485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b169ef-d76c-4f72-a72c-d8cfd39814d7 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.269888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ec4c189693458ab07bde1354c2a439', 'request_id': 'req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:41.275767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ec4c189693458ab07bde1354c2a439', 'request_id': 'req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:41.282558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ec4c189693458ab07bde1354c2a439', 'request_id': 'req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:41.329705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b169ef-d76c-4f72-a72c-d8cfd39814d7 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.358845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] The token cache is being invalidate because project c5ec4c189693458ab07bde1354c2a439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:41.359524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:41.360221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3d927531-c9b3-4957-8afa-2ce2dcd85c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:41.362152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 997/2007] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:41 2022] DELETE /identity/v3/projects/c5ec4c189693458ab07bde1354c2a439 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:41.368457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87282a67-ff79-4f0c-912c-96ce609ba44c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.368859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87282a67-ff79-4f0c-912c-96ce609ba44c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.369605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87282a67-ff79-4f0c-912c-96ce609ba44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.394522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-87282a67-ff79-4f0c-912c-96ce609ba44c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:41.395952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-87282a67-ff79-4f0c-912c-96ce609ba44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:41.401463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0263a-2693-4c7e-9886-b8ef13ee4cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.402251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0263a-2693-4c7e-9886-b8ef13ee4cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.402611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0263a-2693-4c7e-9886-b8ef13ee4cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.402995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0263a-2693-4c7e-9886-b8ef13ee4cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.403766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0263a-2693-4c7e-9886-b8ef13ee4cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.404972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 998/2008] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:41.411650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0263a-2693-4c7e-9886-b8ef13ee4cb6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.423049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1011/2009] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:26:41.425298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22868f12-6b36-4e62-8e22-7fce660e272f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.425691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22868f12-6b36-4e62-8e22-7fce660e272f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.432823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22868f12-6b36-4e62-8e22-7fce660e272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.433930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0263a-2693-4c7e-9886-b8ef13ee4cb6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.437730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-12b0263a-2693-4c7e-9886-b8ef13ee4cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:41.451787 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0263a-2693-4c7e-9886-b8ef13ee4cb6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.460561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-22868f12-6b36-4e62-8e22-7fce660e272f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:41.461946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-22868f12-6b36-4e62-8e22-7fce660e272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:41.462378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f876e9a-f202-4210-ad35-d2b63ad9a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.463136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f876e9a-f202-4210-ad35-d2b63ad9a96f admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.463499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f876e9a-f202-4210-ad35-d2b63ad9a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.463886 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f876e9a-f202-4210-ad35-d2b63ad9a96f admin admin] PATH_INFO: `/v3/projects/67b5ba4ab2034d58905342d90f663692/users/f0967244091c47f1991f90d4548aedd9/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.466047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f876e9a-f202-4210-ad35-d2b63ad9a96f admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b5ba4ab2034d58905342d90f663692, user_id=f0967244091c47f1991f90d4548aedd9, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.468981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f876e9a-f202-4210-ad35-d2b63ad9a96f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.471914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 999/2010] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:41.480664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1012/2011] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:41 2022] PUT /identity/v3/projects/67b5ba4ab2034d58905342d90f663692/users/f0967244091c47f1991f90d4548aedd9/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:41.487026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22868f12-6b36-4e62-8e22-7fce660e272f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.491436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00d445-6897-4228-a7ed-13b90536009e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.491583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00d445-6897-4228-a7ed-13b90536009e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.491834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00d445-6897-4228-a7ed-13b90536009e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.496812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22868f12-6b36-4e62-8e22-7fce660e272f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.527577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1e00d445-6897-4228-a7ed-13b90536009e None None] MFA Rules not processed for user `f0967244091c47f1991f90d4548aedd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:41.529413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1e00d445-6897-4228-a7ed-13b90536009e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:41.563578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa85e01-c657-45c7-9f4f-a55f7a217d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.564563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa85e01-c657-45c7-9f4f-a55f7a217d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.564850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa85e01-c657-45c7-9f4f-a55f7a217d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.565179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa85e01-c657-45c7-9f4f-a55f7a217d0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.565834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa85e01-c657-45c7-9f4f-a55f7a217d0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:41.566789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa85e01-c657-45c7-9f4f-a55f7a217d0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.568210 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa85e01-c657-45c7-9f4f-a55f7a217d0b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.580938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1000/2012] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:26:41.585342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa85e01-c657-45c7-9f4f-a55f7a217d0b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.595632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa85e01-c657-45c7-9f4f-a55f7a217d0b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.600517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f253d7a-d80c-48fb-aea9-963feed30bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.601113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253d7a-d80c-48fb-aea9-963feed30bfa admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.601392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253d7a-d80c-48fb-aea9-963feed30bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.601668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253d7a-d80c-48fb-aea9-963feed30bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.602374 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f253d7a-d80c-48fb-aea9-963feed30bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.602481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1013/2013] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:41.604296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f253d7a-d80c-48fb-aea9-963feed30bfa admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.613659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00d445-6897-4228-a7ed-13b90536009e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.615315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1e00d445-6897-4228-a7ed-13b90536009e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:41.675773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00d445-6897-4228-a7ed-13b90536009e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.679743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425fccc-ac3d-4b94-b2a4-04028cb8a8a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.680380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425fccc-ac3d-4b94-b2a4-04028cb8a8a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.680663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425fccc-ac3d-4b94-b2a4-04028cb8a8a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.680963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425fccc-ac3d-4b94-b2a4-04028cb8a8a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.682819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b425fccc-ac3d-4b94-b2a4-04028cb8a8a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:41.708670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1001/2014] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:41 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:41.712834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f253d7a-d80c-48fb-aea9-963feed30bfa admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.744212 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425fccc-ac3d-4b94-b2a4-04028cb8a8a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.745667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425fccc-ac3d-4b94-b2a4-04028cb8a8a1 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.755651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1014/2015] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:41.771958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:41.776109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f253d7a-d80c-48fb-aea9-963feed30bfa admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.777771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:41.781076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f26d5-c5dc-49bd-ad16-928bdebd571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.781754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f26d5-c5dc-49bd-ad16-928bdebd571a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.782147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f26d5-c5dc-49bd-ad16-928bdebd571a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.782431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f26d5-c5dc-49bd-ad16-928bdebd571a admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.783062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f26d5-c5dc-49bd-ad16-928bdebd571a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.784986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f26d5-c5dc-49bd-ad16-928bdebd571a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.881052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1002/2016] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:41 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:41.885196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425fccc-ac3d-4b94-b2a4-04028cb8a8a1 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.896654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425fccc-ac3d-4b94-b2a4-04028cb8a8a1 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.901330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8663385-e378-482c-b4e5-3a61ec2edf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.901964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8663385-e378-482c-b4e5-3a61ec2edf3d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.902246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8663385-e378-482c-b4e5-3a61ec2edf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.902514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8663385-e378-482c-b4e5-3a61ec2edf3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.903271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8663385-e378-482c-b4e5-3a61ec2edf3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.905494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8663385-e378-482c-b4e5-3a61ec2edf3d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.915730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1015/2017] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:41.922160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f26d5-c5dc-49bd-ad16-928bdebd571a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.941116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f26d5-c5dc-49bd-ad16-928bdebd571a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.946199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc82d3-a52b-4499-a8fa-438a15f4047b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.946812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc82d3-a52b-4499-a8fa-438a15f4047b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.947094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc82d3-a52b-4499-a8fa-438a15f4047b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.947370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc82d3-a52b-4499-a8fa-438a15f4047b admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:41.948020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc82d3-a52b-4499-a8fa-438a15f4047b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:41.949614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc82d3-a52b-4499-a8fa-438a15f4047b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:41.960601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1003/2018] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:41.964816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8663385-e378-482c-b4e5-3a61ec2edf3d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:41.976647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8663385-e378-482c-b4e5-3a61ec2edf3d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:41.982751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632921f-d219-429f-94c9-98328032e3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:41.983414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632921f-d219-429f-94c9-98328032e3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:41.983695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632921f-d219-429f-94c9-98328032e3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:41.983970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632921f-d219-429f-94c9-98328032e3ce admin admin] PATH_INFO: `/v3/projects/34d30dd575254e62beb5aa129625f35f/users/5db2ebc219594630a3ec0bb39f2117e0/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:42.000640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632921f-d219-429f-94c9-98328032e3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d30dd575254e62beb5aa129625f35f, user_id=5db2ebc219594630a3ec0bb39f2117e0, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:42.003381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632921f-d219-429f-94c9-98328032e3ce admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:42.079109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1016/2019] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:41 2022] PUT /identity/v3/projects/34d30dd575254e62beb5aa129625f35f/users/5db2ebc219594630a3ec0bb39f2117e0/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:42.090585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb50a5-12b2-4dd7-9a6f-3cfe09ec49f1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:42.090905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb50a5-12b2-4dd7-9a6f-3cfe09ec49f1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:42.091183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb50a5-12b2-4dd7-9a6f-3cfe09ec49f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:42.119946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-faeb50a5-12b2-4dd7-9a6f-3cfe09ec49f1 None None] MFA Rules not processed for user `5db2ebc219594630a3ec0bb39f2117e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:42.121209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-faeb50a5-12b2-4dd7-9a6f-3cfe09ec49f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:42.186436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1004/2020] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:42.191048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632921f-d219-429f-94c9-98328032e3ce admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:42.205557 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:42.255052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632921f-d219-429f-94c9-98328032e3ce admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:42.259798 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61488078-7b91-48a6-8e5d-99ee424240bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:42.260400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61488078-7b91-48a6-8e5d-99ee424240bd admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:42.260679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61488078-7b91-48a6-8e5d-99ee424240bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:42.260950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61488078-7b91-48a6-8e5d-99ee424240bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:42.261579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61488078-7b91-48a6-8e5d-99ee424240bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:42.263656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61488078-7b91-48a6-8e5d-99ee424240bd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:42.365680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1017/2021] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:42 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:42.369834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb50a5-12b2-4dd7-9a6f-3cfe09ec49f1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:42.381301 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb50a5-12b2-4dd7-9a6f-3cfe09ec49f1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:42.479115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b8ded-24a1-4f98-ba5d-880e3baf136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:42.479740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b8ded-24a1-4f98-ba5d-880e3baf136c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:42.480019 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b8ded-24a1-4f98-ba5d-880e3baf136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:42.480288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b8ded-24a1-4f98-ba5d-880e3baf136c admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:42.481169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b8ded-24a1-4f98-ba5d-880e3baf136c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:42.483062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b8ded-24a1-4f98-ba5d-880e3baf136c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:42.604426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1005/2022] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:42 2022] POST /identity/v3/users => generated 290 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:42.608626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61488078-7b91-48a6-8e5d-99ee424240bd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:42.619718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61488078-7b91-48a6-8e5d-99ee424240bd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:42.624211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e25fcb-cd17-4cdd-881a-e72957953561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:42.624815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e25fcb-cd17-4cdd-881a-e72957953561 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:42.625131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e25fcb-cd17-4cdd-881a-e72957953561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:42.625407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e25fcb-cd17-4cdd-881a-e72957953561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:42.626105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e25fcb-cd17-4cdd-881a-e72957953561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:42.627745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e25fcb-cd17-4cdd-881a-e72957953561 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:42.639423 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1018/2023] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:42.644963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b8ded-24a1-4f98-ba5d-880e3baf136c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:42.655287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b8ded-24a1-4f98-ba5d-880e3baf136c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:42.660573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420d98a-828f-4c8c-b6fe-2af4214cd5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:42.661270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420d98a-828f-4c8c-b6fe-2af4214cd5b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:42.661555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420d98a-828f-4c8c-b6fe-2af4214cd5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:42.661912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420d98a-828f-4c8c-b6fe-2af4214cd5b2 admin admin] PATH_INFO: `/v3/projects/094ddaa85c504b6785cb70c5f1708215/users/b9a65d5dbae34972938bdfcd3c6a2d37/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:42.678184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420d98a-828f-4c8c-b6fe-2af4214cd5b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=094ddaa85c504b6785cb70c5f1708215, user_id=b9a65d5dbae34972938bdfcd3c6a2d37, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:42.680867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420d98a-828f-4c8c-b6fe-2af4214cd5b2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:42.770638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1006/2024] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:42 2022] PUT /identity/v3/projects/094ddaa85c504b6785cb70c5f1708215/users/b9a65d5dbae34972938bdfcd3c6a2d37/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:42.774785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e25fcb-cd17-4cdd-881a-e72957953561 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:42.785112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e25fcb-cd17-4cdd-881a-e72957953561 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:42.841899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd2a06-fd88-4d8f-84a3-8809ed4e1440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:42.842498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd2a06-fd88-4d8f-84a3-8809ed4e1440 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:42.842775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd2a06-fd88-4d8f-84a3-8809ed4e1440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:42.843051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd2a06-fd88-4d8f-84a3-8809ed4e1440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:42.843691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd2a06-fd88-4d8f-84a3-8809ed4e1440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:42.845291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd2a06-fd88-4d8f-84a3-8809ed4e1440 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:42.855125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1019/2025] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:42.859353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420d98a-828f-4c8c-b6fe-2af4214cd5b2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:42.870144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420d98a-828f-4c8c-b6fe-2af4214cd5b2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:42.875961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c7cc0-1e53-4505-819a-e308e66e78ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:42.876887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c7cc0-1e53-4505-819a-e308e66e78ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:42.877160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c7cc0-1e53-4505-819a-e308e66e78ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:42.877431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c7cc0-1e53-4505-819a-e308e66e78ab admin admin] PATH_INFO: `/v3/projects/094ddaa85c504b6785cb70c5f1708215/users/b9a65d5dbae34972938bdfcd3c6a2d37/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:42.879395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c7cc0-1e53-4505-819a-e308e66e78ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=094ddaa85c504b6785cb70c5f1708215, user_id=b9a65d5dbae34972938bdfcd3c6a2d37, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:42.882068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c7cc0-1e53-4505-819a-e308e66e78ab admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:42.893418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1007/2026] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:42 2022] PUT /identity/v3/projects/094ddaa85c504b6785cb70c5f1708215/users/b9a65d5dbae34972938bdfcd3c6a2d37/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:42.903840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda44843-6c39-41ed-8103-9bed415a2472 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:42.904133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda44843-6c39-41ed-8103-9bed415a2472 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:42.904410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda44843-6c39-41ed-8103-9bed415a2472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:42.932351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bda44843-6c39-41ed-8103-9bed415a2472 None None] MFA Rules not processed for user `b9a65d5dbae34972938bdfcd3c6a2d37`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:42.933648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bda44843-6c39-41ed-8103-9bed415a2472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:43.006102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1020/2027] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:43.018032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c7cc0-1e53-4505-819a-e308e66e78ab admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:43.019590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-674c7cc0-1e53-4505-819a-e308e66e78ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:43.042339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:43.079392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c7cc0-1e53-4505-819a-e308e66e78ab admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:43.083096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb781d-9c6e-4ecc-bd42-397163d530cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:43.083743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb781d-9c6e-4ecc-bd42-397163d530cf service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:43.084327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb781d-9c6e-4ecc-bd42-397163d530cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:43.084779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb781d-9c6e-4ecc-bd42-397163d530cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:43.094476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb781d-9c6e-4ecc-bd42-397163d530cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:43.095885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb781d-9c6e-4ecc-bd42-397163d530cf service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:43.106063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1008/2028] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:43.211040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda44843-6c39-41ed-8103-9bed415a2472 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:43.220840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda44843-6c39-41ed-8103-9bed415a2472 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:43.224471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a6139-52c5-46fb-bd78-a1895807246a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:43.225083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a6139-52c5-46fb-bd78-a1895807246a service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:43.225359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a6139-52c5-46fb-bd78-a1895807246a service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:43.225630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a6139-52c5-46fb-bd78-a1895807246a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:43.232310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:43.284261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a6139-52c5-46fb-bd78-a1895807246a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:43.285904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a6139-52c5-46fb-bd78-a1895807246a service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:43.298228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1021/2029] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:47.712732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036e755-2cb5-494e-910c-6e3c283f0074 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:47.713287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036e755-2cb5-494e-910c-6e3c283f0074 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:47.713287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036e755-2cb5-494e-910c-6e3c283f0074 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:47.714905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1009/2030] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:26:47.726650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f02527-bcc4-4dc6-8076-fcba6cf98b50 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:47.727158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f02527-bcc4-4dc6-8076-fcba6cf98b50 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:47.727397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f02527-bcc4-4dc6-8076-fcba6cf98b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:47.748329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-16f02527-bcc4-4dc6-8076-fcba6cf98b50 None None] MFA Rules not processed for user `eb76ad667a744126ab00b38c346e9017`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:47.749563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-16f02527-bcc4-4dc6-8076-fcba6cf98b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:47.814691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1022/2031] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:26:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:47.825607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036e755-2cb5-494e-910c-6e3c283f0074 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:47.827308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3036e755-2cb5-494e-910c-6e3c283f0074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:47.843948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036e755-2cb5-494e-910c-6e3c283f0074 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:47.849482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af67113-38ad-4e68-837b-ad137bee005f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:47.850246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af67113-38ad-4e68-837b-ad137bee005f service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:47.850545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af67113-38ad-4e68-837b-ad137bee005f service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:47.850803 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af67113-38ad-4e68-837b-ad137bee005f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:47.860258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af67113-38ad-4e68-837b-ad137bee005f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:47.861859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af67113-38ad-4e68-837b-ad137bee005f service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:47.877386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1010/2032] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:47.884720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f02527-bcc4-4dc6-8076-fcba6cf98b50 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:47.886620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-16f02527-bcc4-4dc6-8076-fcba6cf98b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:47.964558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f02527-bcc4-4dc6-8076-fcba6cf98b50 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:47.972995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:47.973641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:47.973958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:47.974231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] PATH_INFO: `/v3/users/4aa39f218d4d463291ffe15b929c4e2d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:47.975401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa39f218d4d463291ffe15b929c4e2d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:47.977344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:48.125524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:48.127121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa39f218d4d463291ffe15b929c4e2d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:48.127804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa39f218d4d463291ffe15b929c4e2d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:48.128513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa39f218d4d463291ffe15b929c4e2d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:48.135515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa39f218d4d463291ffe15b929c4e2d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:48.147937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa39f218d4d463291ffe15b929c4e2d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:48.228350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1023/2033] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:47 2022] DELETE /identity/v3/users/4aa39f218d4d463291ffe15b929c4e2d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:48.481328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af67113-38ad-4e68-837b-ad137bee005f service placement] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:48.496590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af67113-38ad-4e68-837b-ad137bee005f service placement] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:48.559542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:48.560131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:48.560417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:48.560691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] PATH_INFO: `/v3/projects/b4b6d46119374e6ab76032a4db1198d5` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:48.561832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4b6d46119374e6ab76032a4db1198d5)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:48.563858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:48.954217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:48.954217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:48.967208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d77c0-1d18-4696-ac49-fa5bb961eb41 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:48.972020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:48.972652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:48.972931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:48.973201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] PATH_INFO: `/v3/users/5db2ebc219594630a3ec0bb39f2117e0` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:48.974344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db2ebc219594630a3ec0bb39f2117e0)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:48.976270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:49.006684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4b6d46119374e6ab76032a4db1198d5', 'request_id': 'req-153455d5-da0c-464c-8747-cb4d2385ebb4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.012806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4b6d46119374e6ab76032a4db1198d5', 'request_id': 'req-153455d5-da0c-464c-8747-cb4d2385ebb4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.019490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4b6d46119374e6ab76032a4db1198d5', 'request_id': 'req-153455d5-da0c-464c-8747-cb4d2385ebb4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.118625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] The token cache is being invalidate because project b4b6d46119374e6ab76032a4db1198d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:49.118919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.119563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-153455d5-da0c-464c-8747-cb4d2385ebb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.121109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1011/2034] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:48 2022] DELETE /identity/v3/projects/b4b6d46119374e6ab76032a4db1198d5 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:49.126053 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec4b60-82fd-4983-b65b-d06a108299a8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.130136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec4b60-82fd-4983-b65b-d06a108299a8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.130136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec4b60-82fd-4983-b65b-d06a108299a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.147886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-55ec4b60-82fd-4983-b65b-d06a108299a8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:49.149101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-55ec4b60-82fd-4983-b65b-d06a108299a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:49.183901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:49.210847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1012/2035] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:49.219290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec72524-607c-4350-8bf9-1cb156395708 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.219581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec72524-607c-4350-8bf9-1cb156395708 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.219856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec72524-607c-4350-8bf9-1cb156395708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.221589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db2ebc219594630a3ec0bb39f2117e0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.222520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db2ebc219594630a3ec0bb39f2117e0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.231712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db2ebc219594630a3ec0bb39f2117e0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.232554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db2ebc219594630a3ec0bb39f2117e0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.244314 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cec72524-607c-4350-8bf9-1cb156395708 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:49.245565 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cec72524-607c-4350-8bf9-1cb156395708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:49.248998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ea01909-aa6a-406c-9cb8-da0395d590bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db2ebc219594630a3ec0bb39f2117e0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:49.256372 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1013/2036] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:49.264187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec72524-607c-4350-8bf9-1cb156395708 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:49.273928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec72524-607c-4350-8bf9-1cb156395708 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:49.279307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d6125-287a-4b93-a031-a1fb4cb02e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:49.279910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d6125-287a-4b93-a031-a1fb4cb02e95 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.280187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d6125-287a-4b93-a031-a1fb4cb02e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.280458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d6125-287a-4b93-a031-a1fb4cb02e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.281112 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d6125-287a-4b93-a031-a1fb4cb02e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:49.281410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d6125-287a-4b93-a031-a1fb4cb02e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:49.283144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d6125-287a-4b93-a031-a1fb4cb02e95 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:49.295506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1014/2037] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:26:49.301165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d6125-287a-4b93-a031-a1fb4cb02e95 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:49.311483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d6125-287a-4b93-a031-a1fb4cb02e95 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:49.316273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c2ef4-95b6-497f-968b-47b77df0e9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:49.316873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2ef4-95b6-497f-968b-47b77df0e9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.317151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2ef4-95b6-497f-968b-47b77df0e9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.317424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2ef4-95b6-497f-968b-47b77df0e9b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.318082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c2ef4-95b6-497f-968b-47b77df0e9b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:49.320033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c2ef4-95b6-497f-968b-47b77df0e9b4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:49.336661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1024/2038] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:48 2022] DELETE /identity/v3/users/5db2ebc219594630a3ec0bb39f2117e0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:49.341732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e076072-96ba-4892-9aa8-ab43ffcfc1fe None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.342069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e076072-96ba-4892-9aa8-ab43ffcfc1fe None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.342344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e076072-96ba-4892-9aa8-ab43ffcfc1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.361013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9e076072-96ba-4892-9aa8-ab43ffcfc1fe None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:49.362341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9e076072-96ba-4892-9aa8-ab43ffcfc1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:49.423594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1025/2039] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:49.433538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e076072-96ba-4892-9aa8-ab43ffcfc1fe None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:49.435025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9e076072-96ba-4892-9aa8-ab43ffcfc1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:49.458207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1015/2040] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:49 2022] POST /identity/v3/projects => generated 393 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:49.464081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c2ef4-95b6-497f-968b-47b77df0e9b4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:49.476558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:49.493570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e076072-96ba-4892-9aa8-ab43ffcfc1fe None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:49.530143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c2ef4-95b6-497f-968b-47b77df0e9b4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:49.535492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a350ec-b0e6-4799-a7c0-4c4a04662c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:49.536106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a350ec-b0e6-4799-a7c0-4c4a04662c48 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.536381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a350ec-b0e6-4799-a7c0-4c4a04662c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.536647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a350ec-b0e6-4799-a7c0-4c4a04662c48 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.537538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a350ec-b0e6-4799-a7c0-4c4a04662c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:49.539496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a350ec-b0e6-4799-a7c0-4c4a04662c48 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:49.545924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92369704-7caf-4622-ae85-9d5e876624c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:49.546534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92369704-7caf-4622-ae85-9d5e876624c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.546809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92369704-7caf-4622-ae85-9d5e876624c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.547077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92369704-7caf-4622-ae85-9d5e876624c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.556850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92369704-7caf-4622-ae85-9d5e876624c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:49.558329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92369704-7caf-4622-ae85-9d5e876624c5 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:49.568387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1026/2041] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:49.635819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1016/2042] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:49 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:49.641536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92369704-7caf-4622-ae85-9d5e876624c5 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:49.655678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92369704-7caf-4622-ae85-9d5e876624c5 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:49.662578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb398b5-3e5e-48af-8671-5c83f806e755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:49.663280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb398b5-3e5e-48af-8671-5c83f806e755 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.663654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb398b5-3e5e-48af-8671-5c83f806e755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.664003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb398b5-3e5e-48af-8671-5c83f806e755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.664737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb398b5-3e5e-48af-8671-5c83f806e755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:49.666462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb398b5-3e5e-48af-8671-5c83f806e755 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:49.683171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1027/2043] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:49.690105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a350ec-b0e6-4799-a7c0-4c4a04662c48 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:49.701502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a350ec-b0e6-4799-a7c0-4c4a04662c48 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:49.707622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa3ec9-6d2d-4722-a694-956b9ed16380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:49.707972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa3ec9-6d2d-4722-a694-956b9ed16380 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.708252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa3ec9-6d2d-4722-a694-956b9ed16380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.708527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa3ec9-6d2d-4722-a694-956b9ed16380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.709187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa3ec9-6d2d-4722-a694-956b9ed16380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:49.722540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa3ec9-6d2d-4722-a694-956b9ed16380 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:49.722691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1017/2044] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:49.727001 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb398b5-3e5e-48af-8671-5c83f806e755 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:49.738113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb398b5-3e5e-48af-8671-5c83f806e755 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:49.743066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70a78a-20a2-4dc0-bf6d-41cc8bed545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:49.743728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70a78a-20a2-4dc0-bf6d-41cc8bed545a admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.744031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70a78a-20a2-4dc0-bf6d-41cc8bed545a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.744280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70a78a-20a2-4dc0-bf6d-41cc8bed545a admin admin] PATH_INFO: `/v3/projects/e2a13f3ca60643a9a3507476325d351a/users/57be68e76fc14d2ca309571e284509bd/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.759654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70a78a-20a2-4dc0-bf6d-41cc8bed545a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2a13f3ca60643a9a3507476325d351a, user_id=57be68e76fc14d2ca309571e284509bd, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:49.762437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70a78a-20a2-4dc0-bf6d-41cc8bed545a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:49.870049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1028/2045] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:49 2022] PUT /identity/v3/projects/e2a13f3ca60643a9a3507476325d351a/users/57be68e76fc14d2ca309571e284509bd/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:49.875542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955adfba-e018-4549-938d-0b0319df637c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:49.875832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955adfba-e018-4549-938d-0b0319df637c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:49.876110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955adfba-e018-4549-938d-0b0319df637c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:49.897696 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-955adfba-e018-4549-938d-0b0319df637c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:49.898960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-955adfba-e018-4549-938d-0b0319df637c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:49.962265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1018/2046] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:49.973078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70a78a-20a2-4dc0-bf6d-41cc8bed545a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:50.046598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70a78a-20a2-4dc0-bf6d-41cc8bed545a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:50.046702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955adfba-e018-4549-938d-0b0319df637c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:50.047115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635f1d9-bb71-4961-adc9-d924fba33498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:50.047219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-955adfba-e018-4549-938d-0b0319df637c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:50.047295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635f1d9-bb71-4961-adc9-d924fba33498 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:50.047636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635f1d9-bb71-4961-adc9-d924fba33498 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:50.048055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635f1d9-bb71-4961-adc9-d924fba33498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:50.060120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635f1d9-bb71-4961-adc9-d924fba33498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:50.061784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635f1d9-bb71-4961-adc9-d924fba33498 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:50.062564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955adfba-e018-4549-938d-0b0319df637c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:50.067572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:50.068221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:50.068505 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:50.068782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] PATH_INFO: `/v3/projects/34d30dd575254e62beb5aa129625f35f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:50.069997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d30dd575254e62beb5aa129625f35f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:50.072268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:50.073591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1029/2047] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:50.086273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:50.253440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d30dd575254e62beb5aa129625f35f', 'request_id': 'req-bcbda5d8-eff8-4552-ae72-4ec386e6987e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.261086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d30dd575254e62beb5aa129625f35f', 'request_id': 'req-bcbda5d8-eff8-4552-ae72-4ec386e6987e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.268524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d30dd575254e62beb5aa129625f35f', 'request_id': 'req-bcbda5d8-eff8-4552-ae72-4ec386e6987e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.369239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] The token cache is being invalidate because project 34d30dd575254e62beb5aa129625f35f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:50.374863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.374863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.374863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1019/2048] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:50 2022] DELETE /identity/v3/projects/34d30dd575254e62beb5aa129625f35f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:50.376397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635f1d9-bb71-4961-adc9-d924fba33498 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:50.386200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635f1d9-bb71-4961-adc9-d924fba33498 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:50.445854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:50.446525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:50.446806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:50.447084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] PATH_INFO: `/v3/users/b9a65d5dbae34972938bdfcd3c6a2d37` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:50.448309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a65d5dbae34972938bdfcd3c6a2d37)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:50.450974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:50.458846 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:50.747743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a65d5dbae34972938bdfcd3c6a2d37'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.748420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a65d5dbae34972938bdfcd3c6a2d37'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.749024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a65d5dbae34972938bdfcd3c6a2d37'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.767287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a65d5dbae34972938bdfcd3c6a2d37'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.780732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c7aa05fc-d763-4d9d-ab0e-3ea8f527c2f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a65d5dbae34972938bdfcd3c6a2d37'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:50.875819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1030/2049] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:50 2022] DELETE /identity/v3/users/b9a65d5dbae34972938bdfcd3c6a2d37 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:51.392157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:51.394630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:51.410101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbda5d8-eff8-4552-ae72-4ec386e6987e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:51.476024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:51.476657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:51.476963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:51.477235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] PATH_INFO: `/v3/projects/094ddaa85c504b6785cb70c5f1708215` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:51.478394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] RBAC: Authorizing `identity:delete_project(project_id=094ddaa85c504b6785cb70c5f1708215)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:51.480298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:51.597601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094ddaa85c504b6785cb70c5f1708215', 'request_id': 'req-7e5e4d7c-2272-4950-82ed-8a65d759301f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:51.603383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094ddaa85c504b6785cb70c5f1708215', 'request_id': 'req-7e5e4d7c-2272-4950-82ed-8a65d759301f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:51.610813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094ddaa85c504b6785cb70c5f1708215', 'request_id': 'req-7e5e4d7c-2272-4950-82ed-8a65d759301f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:51.698382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] The token cache is being invalidate because project 094ddaa85c504b6785cb70c5f1708215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:51.698682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:51.699450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7e5e4d7c-2272-4950-82ed-8a65d759301f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:51.704538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1020/2050] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:51 2022] DELETE /identity/v3/projects/094ddaa85c504b6785cb70c5f1708215 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:51.706359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d7cff-d2de-43ac-854a-34b564e536b0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:51.706781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d7cff-d2de-43ac-854a-34b564e536b0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:51.706879 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d7cff-d2de-43ac-854a-34b564e536b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:51.739264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9c9d7cff-d2de-43ac-854a-34b564e536b0 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:51.741196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9c9d7cff-d2de-43ac-854a-34b564e536b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:51.859550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1031/2051] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:51.868039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f53993-2ce6-491e-8bbf-07660043f3f1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:51.868952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f53993-2ce6-491e-8bbf-07660043f3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:51.869618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f53993-2ce6-491e-8bbf-07660043f3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:51.889495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-38f53993-2ce6-491e-8bbf-07660043f3f1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:51.890864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-38f53993-2ce6-491e-8bbf-07660043f3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:51.901909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1021/2052] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:51.910984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d7cff-d2de-43ac-854a-34b564e536b0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:51.930261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d7cff-d2de-43ac-854a-34b564e536b0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:51.930261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b39f-a440-422f-8eff-cb5b3f177783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:51.930261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b39f-a440-422f-8eff-cb5b3f177783 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:51.930261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b39f-a440-422f-8eff-cb5b3f177783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:51.930560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b39f-a440-422f-8eff-cb5b3f177783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:51.931358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115b39f-a440-422f-8eff-cb5b3f177783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:51.931853 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115b39f-a440-422f-8eff-cb5b3f177783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:51.933841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115b39f-a440-422f-8eff-cb5b3f177783 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:51.952426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1032/2053] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:26:51.956963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f53993-2ce6-491e-8bbf-07660043f3f1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:51.968094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f53993-2ce6-491e-8bbf-07660043f3f1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:51.974022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb216-ed67-4501-aad3-e4cc9f550afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:51.976032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb216-ed67-4501-aad3-e4cc9f550afa admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:51.976032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb216-ed67-4501-aad3-e4cc9f550afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:51.976032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb216-ed67-4501-aad3-e4cc9f550afa admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:51.976032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647eb216-ed67-4501-aad3-e4cc9f550afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:51.977703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647eb216-ed67-4501-aad3-e4cc9f550afa admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:52.176171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:52.182838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1022/2054] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:51 2022] POST /identity/v3/projects => generated 391 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:52.189080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b39f-a440-422f-8eff-cb5b3f177783 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:52.199545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b39f-a440-422f-8eff-cb5b3f177783 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:52.255815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3707c-197b-46c5-ad56-ea1540a053d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:52.256424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3707c-197b-46c5-ad56-ea1540a053d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:52.256721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3707c-197b-46c5-ad56-ea1540a053d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:52.256982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3707c-197b-46c5-ad56-ea1540a053d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:52.257616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3707c-197b-46c5-ad56-ea1540a053d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:52.263521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3707c-197b-46c5-ad56-ea1540a053d3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:52.307062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb216-ed67-4501-aad3-e4cc9f550afa admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:52.308805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-647eb216-ed67-4501-aad3-e4cc9f550afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:52.326506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb216-ed67-4501-aad3-e4cc9f550afa admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:52.327468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:52.328109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:52.328918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:52.328918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] PATH_INFO: `/v3/users/3240f87bae9a448e9661e3228fb3e9c6` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:52.329921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3240f87bae9a448e9661e3228fb3e9c6)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:52.331793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:52.376353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:52.383927 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1033/2055] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:52 2022] POST /identity/v3/users => generated 296 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:26:52.388693 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3707c-197b-46c5-ad56-ea1540a053d3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:52.400022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3707c-197b-46c5-ad56-ea1540a053d3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:52.405241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a775a-acf7-4269-95f5-7fdf4b8bf8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:52.406729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a775a-acf7-4269-95f5-7fdf4b8bf8fa admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:52.406729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a775a-acf7-4269-95f5-7fdf4b8bf8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:52.406729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a775a-acf7-4269-95f5-7fdf4b8bf8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:52.407517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a775a-acf7-4269-95f5-7fdf4b8bf8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:52.409315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a775a-acf7-4269-95f5-7fdf4b8bf8fa admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:52.423899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1034/2056] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:26:52.427954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a775a-acf7-4269-95f5-7fdf4b8bf8fa admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:52.441498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a775a-acf7-4269-95f5-7fdf4b8bf8fa admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:52.445186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b9b20-252d-400d-bf68-8483349305b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:52.445807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9b20-252d-400d-bf68-8483349305b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:52.446124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9b20-252d-400d-bf68-8483349305b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:52.446399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9b20-252d-400d-bf68-8483349305b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:52.447218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b9b20-252d-400d-bf68-8483349305b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:52.449237 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b9b20-252d-400d-bf68-8483349305b2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:52.462371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1035/2057] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:26:52.467241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b9b20-252d-400d-bf68-8483349305b2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:52.478789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b9b20-252d-400d-bf68-8483349305b2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:52.484621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0203c7-eea3-4d02-b2c4-d026ec13a413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:52.485480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0203c7-eea3-4d02-b2c4-d026ec13a413 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:52.485777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0203c7-eea3-4d02-b2c4-d026ec13a413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:52.486094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0203c7-eea3-4d02-b2c4-d026ec13a413 admin admin] PATH_INFO: `/v3/projects/85eeb65090dc48dcb1bead1b1b1422e0/users/6fa21aafc8704e0fb8321363598a040b/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:52.508248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0203c7-eea3-4d02-b2c4-d026ec13a413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85eeb65090dc48dcb1bead1b1b1422e0, user_id=6fa21aafc8704e0fb8321363598a040b, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:52.511172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0203c7-eea3-4d02-b2c4-d026ec13a413 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:52.562942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3240f87bae9a448e9661e3228fb3e9c6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:52.564264 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3240f87bae9a448e9661e3228fb3e9c6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:52.564884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3240f87bae9a448e9661e3228fb3e9c6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:52.628791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1036/2058] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:52 2022] PUT /identity/v3/projects/85eeb65090dc48dcb1bead1b1b1422e0/users/6fa21aafc8704e0fb8321363598a040b/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:52.640367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa474e7-f51e-482e-9279-19e5671196c5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:52.640702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa474e7-f51e-482e-9279-19e5671196c5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:52.640979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa474e7-f51e-482e-9279-19e5671196c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:52.674129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2fa474e7-f51e-482e-9279-19e5671196c5 None None] MFA Rules not processed for user `6fa21aafc8704e0fb8321363598a040b`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:52.675369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2fa474e7-f51e-482e-9279-19e5671196c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:52.679361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3240f87bae9a448e9661e3228fb3e9c6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:52.691378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e32400eb-8cba-441f-90e4-b17d550b619c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3240f87bae9a448e9661e3228fb3e9c6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:52.753814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1023/2059] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:52 2022] DELETE /identity/v3/users/3240f87bae9a448e9661e3228fb3e9c6 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:52.758401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7cd66-6158-40d2-a6a7-ff65c1281348 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:52.759167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7cd66-6158-40d2-a6a7-ff65c1281348 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:52.759708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7cd66-6158-40d2-a6a7-ff65c1281348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:52.763952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1037/2060] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:52.768231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa474e7-f51e-482e-9279-19e5671196c5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:52.770312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2fa474e7-f51e-482e-9279-19e5671196c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:52.796490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-22a7cd66-6158-40d2-a6a7-ff65c1281348 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:52.797795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-22a7cd66-6158-40d2-a6a7-ff65c1281348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:52.878296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa474e7-f51e-482e-9279-19e5671196c5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:52.880615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1024/2061] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:52.893179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7cd66-6158-40d2-a6a7-ff65c1281348 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:52.895195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-22a7cd66-6158-40d2-a6a7-ff65c1281348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:52.896249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e521e37-cfe6-4177-9994-ea8f3cc65794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:52.897295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e521e37-cfe6-4177-9994-ea8f3cc65794 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:52.897596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e521e37-cfe6-4177-9994-ea8f3cc65794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:52.897909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e521e37-cfe6-4177-9994-ea8f3cc65794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:52.898540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e521e37-cfe6-4177-9994-ea8f3cc65794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:52.900783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e521e37-cfe6-4177-9994-ea8f3cc65794 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:52.982784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7cd66-6158-40d2-a6a7-ff65c1281348 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:52.986837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce166-ebb7-4751-a309-46eaf396ec43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:52.987551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce166-ebb7-4751-a309-46eaf396ec43 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:52.987894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce166-ebb7-4751-a309-46eaf396ec43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:52.988230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce166-ebb7-4751-a309-46eaf396ec43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:52.998966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bce166-ebb7-4751-a309-46eaf396ec43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:53.000685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bce166-ebb7-4751-a309-46eaf396ec43 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:53.011876 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1025/2062] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:53.037461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1038/2063] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:52 2022] POST /identity/v3/projects => generated 393 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:53.041416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce166-ebb7-4751-a309-46eaf396ec43 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:53.055840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce166-ebb7-4751-a309-46eaf396ec43 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:53.173239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df8b2f-8dd5-46bf-9b60-c9c47d6f809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:53.174491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df8b2f-8dd5-46bf-9b60-c9c47d6f809a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:53.174773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df8b2f-8dd5-46bf-9b60-c9c47d6f809a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:53.175051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df8b2f-8dd5-46bf-9b60-c9c47d6f809a admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:53.175932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df8b2f-8dd5-46bf-9b60-c9c47d6f809a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:53.177714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df8b2f-8dd5-46bf-9b60-c9c47d6f809a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:53.316111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1026/2064] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:53 2022] POST /identity/v3/users => generated 303 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:53.325633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e521e37-cfe6-4177-9994-ea8f3cc65794 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:53.340156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e521e37-cfe6-4177-9994-ea8f3cc65794 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:53.349297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a15dc-91fb-4a09-97b1-c87e78e73f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:53.349297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a15dc-91fb-4a09-97b1-c87e78e73f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:53.349297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a15dc-91fb-4a09-97b1-c87e78e73f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:53.349297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a15dc-91fb-4a09-97b1-c87e78e73f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:53.349693 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a15dc-91fb-4a09-97b1-c87e78e73f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:53.350053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a15dc-91fb-4a09-97b1-c87e78e73f33 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:53.364447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1039/2065] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:53.368916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df8b2f-8dd5-46bf-9b60-c9c47d6f809a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:53.380609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df8b2f-8dd5-46bf-9b60-c9c47d6f809a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:53.386131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee6b3b-b988-4fbc-b2e0-5c837be49469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:53.386859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee6b3b-b988-4fbc-b2e0-5c837be49469 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:53.387256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee6b3b-b988-4fbc-b2e0-5c837be49469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:53.388115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee6b3b-b988-4fbc-b2e0-5c837be49469 admin admin] PATH_INFO: `/v3/projects/0ed5ae1d0d5c4522a5c4921539f801fa/users/42fc023146ed4a669d1daaed19330e07/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:53.410544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee6b3b-b988-4fbc-b2e0-5c837be49469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed5ae1d0d5c4522a5c4921539f801fa, user_id=42fc023146ed4a669d1daaed19330e07, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:53.410544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee6b3b-b988-4fbc-b2e0-5c837be49469 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:53.495278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1027/2066] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:53 2022] PUT /identity/v3/projects/0ed5ae1d0d5c4522a5c4921539f801fa/users/42fc023146ed4a669d1daaed19330e07/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:53.499631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a15dc-91fb-4a09-97b1-c87e78e73f33 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:53.510996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a15dc-91fb-4a09-97b1-c87e78e73f33 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:53.562240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee6b3b-b988-4fbc-b2e0-5c837be49469 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:53.565835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f2ee6b3b-b988-4fbc-b2e0-5c837be49469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:53.604417 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fa4d6-92b3-474a-82cb-fe6c5dfadb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:53.605674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fa4d6-92b3-474a-82cb-fe6c5dfadb34 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:53.606576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fa4d6-92b3-474a-82cb-fe6c5dfadb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:53.606802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fa4d6-92b3-474a-82cb-fe6c5dfadb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:53.607498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fa4d6-92b3-474a-82cb-fe6c5dfadb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:53.609155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fa4d6-92b3-474a-82cb-fe6c5dfadb34 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:53.620976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1040/2067] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:53.626198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fa4d6-92b3-474a-82cb-fe6c5dfadb34 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:53.649068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fa4d6-92b3-474a-82cb-fe6c5dfadb34 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:53.655703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb7b57-2388-4d30-ad2d-1a50edbb0592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:53.656168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb7b57-2388-4d30-ad2d-1a50edbb0592 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:53.656450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb7b57-2388-4d30-ad2d-1a50edbb0592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:53.656728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb7b57-2388-4d30-ad2d-1a50edbb0592 admin admin] PATH_INFO: `/v3/projects/0ed5ae1d0d5c4522a5c4921539f801fa/users/42fc023146ed4a669d1daaed19330e07/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:53.661283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb7b57-2388-4d30-ad2d-1a50edbb0592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed5ae1d0d5c4522a5c4921539f801fa, user_id=42fc023146ed4a669d1daaed19330e07, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:53.665061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb7b57-2388-4d30-ad2d-1a50edbb0592 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:53.674516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee6b3b-b988-4fbc-b2e0-5c837be49469 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:53.687105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1041/2068] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:53 2022] PUT /identity/v3/projects/0ed5ae1d0d5c4522a5c4921539f801fa/users/42fc023146ed4a669d1daaed19330e07/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:53.692075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120553be-ae44-4bd9-a4bb-9fcc8e29b0ec None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:53.696482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120553be-ae44-4bd9-a4bb-9fcc8e29b0ec None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:53.696482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120553be-ae44-4bd9-a4bb-9fcc8e29b0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:53.745586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-120553be-ae44-4bd9-a4bb-9fcc8e29b0ec None None] MFA Rules not processed for user `42fc023146ed4a669d1daaed19330e07`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:53.746955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-120553be-ae44-4bd9-a4bb-9fcc8e29b0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:53.767873 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:53.768847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:53.769137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:53.769417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] PATH_INFO: `/v3/projects/ce8c5144f1b440f2b33dbca10016d56e` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:53.770674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce8c5144f1b440f2b33dbca10016d56e)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:53.782056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:53.858375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1042/2069] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:53 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:53.869599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120553be-ae44-4bd9-a4bb-9fcc8e29b0ec None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:53.871930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-120553be-ae44-4bd9-a4bb-9fcc8e29b0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:53.917850 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8c5144f1b440f2b33dbca10016d56e', 'request_id': 'req-2cbfd40c-e68d-4604-99f4-82045b8970d5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:53.926537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:53.928563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8c5144f1b440f2b33dbca10016d56e', 'request_id': 'req-2cbfd40c-e68d-4604-99f4-82045b8970d5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:53.941180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8c5144f1b440f2b33dbca10016d56e', 'request_id': 'req-2cbfd40c-e68d-4604-99f4-82045b8970d5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:53.950659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120553be-ae44-4bd9-a4bb-9fcc8e29b0ec None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:54.005024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdde8a-98b1-4830-97b2-682988504a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:54.005643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdde8a-98b1-4830-97b2-682988504a0e service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:54.006208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdde8a-98b1-4830-97b2-682988504a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:54.008617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:54.009289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdde8a-98b1-4830-97b2-682988504a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:54.043986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] The token cache is being invalidate because project ce8c5144f1b440f2b33dbca10016d56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:54.046359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:54.047987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:54.055420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1028/2070] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:53 2022] DELETE /identity/v3/projects/ce8c5144f1b440f2b33dbca10016d56e => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:54.064438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:54.066471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:54.095226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdde8a-98b1-4830-97b2-682988504a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:54.097979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdde8a-98b1-4830-97b2-682988504a0e service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:54.110127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1043/2071] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:53 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:54.158827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfd40c-e68d-4604-99f4-82045b8970d5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:54.162266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdde8a-98b1-4830-97b2-682988504a0e service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:54.163892 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e1fdde8a-98b1-4830-97b2-682988504a0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:54.164620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:54.165273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:54.165568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:54.165832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] PATH_INFO: `/v3/users/b96e86ac3b28459bab1b4353d0ba018c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:54.167190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b96e86ac3b28459bab1b4353d0ba018c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:54.169291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:54.178432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdde8a-98b1-4830-97b2-682988504a0e service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:54.185558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea58b-e829-430a-be46-d055d640129d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:54.187278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea58b-e829-430a-be46-d055d640129d service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:54.187765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea58b-e829-430a-be46-d055d640129d service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:54.188232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea58b-e829-430a-be46-d055d640129d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:54.191600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-78cea58b-e829-430a-be46-d055d640129d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:54.270229 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cea58b-e829-430a-be46-d055d640129d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:54.274902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cea58b-e829-430a-be46-d055d640129d service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:54.291633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1044/2072] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:54.359634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b96e86ac3b28459bab1b4353d0ba018c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:54.360390 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b96e86ac3b28459bab1b4353d0ba018c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:54.361123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b96e86ac3b28459bab1b4353d0ba018c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:54.416199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b96e86ac3b28459bab1b4353d0ba018c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:54.427300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-94840e52-780c-4dbf-99c8-207a2d3f4617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b96e86ac3b28459bab1b4353d0ba018c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:54.453844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1029/2073] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:54 2022] DELETE /identity/v3/users/b96e86ac3b28459bab1b4353d0ba018c => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:54.932489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea58b-e829-430a-be46-d055d640129d service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:54.944213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea58b-e829-430a-be46-d055d640129d service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:55.013142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:55.013914 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:55.014166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:55.015952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] PATH_INFO: `/v3/projects/826d165304764abc8b879fa4e5273fa9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:55.015952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=826d165304764abc8b879fa4e5273fa9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:55.017687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:55.253998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '826d165304764abc8b879fa4e5273fa9', 'request_id': 'req-b5de0c21-744d-42fd-a551-5446d0038609'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:55.265718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '826d165304764abc8b879fa4e5273fa9', 'request_id': 'req-b5de0c21-744d-42fd-a551-5446d0038609'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:55.358884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '826d165304764abc8b879fa4e5273fa9', 'request_id': 'req-b5de0c21-744d-42fd-a551-5446d0038609'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:55.370947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] The token cache is being invalidate because project 826d165304764abc8b879fa4e5273fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:55.371257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:55.371991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5de0c21-744d-42fd-a551-5446d0038609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:55.373481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1045/2074] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:54 2022] DELETE /identity/v3/projects/826d165304764abc8b879fa4e5273fa9 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:55.378395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff1a6-1815-4a8d-831e-3c7920ff6d12 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:55.378882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff1a6-1815-4a8d-831e-3c7920ff6d12 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:55.378973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff1a6-1815-4a8d-831e-3c7920ff6d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:55.398917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2d9ff1a6-1815-4a8d-831e-3c7920ff6d12 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:55.400121 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2d9ff1a6-1815-4a8d-831e-3c7920ff6d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:55.477585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9431e-55bb-4e50-ac3e-fa94288f1919 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:55.477585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9431e-55bb-4e50-ac3e-fa94288f1919 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:55.477585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9431e-55bb-4e50-ac3e-fa94288f1919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:55.478066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1030/2075] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:55.511298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-faa9431e-55bb-4e50-ac3e-fa94288f1919 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:55.512928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-faa9431e-55bb-4e50-ac3e-fa94288f1919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:55.522571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1046/2076] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:55.534871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff1a6-1815-4a8d-831e-3c7920ff6d12 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:55.544487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff1a6-1815-4a8d-831e-3c7920ff6d12 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:55.550573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd3589-39f9-4b7b-96ec-23cd4d68fbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:55.552314 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd3589-39f9-4b7b-96ec-23cd4d68fbdf admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:55.553178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd3589-39f9-4b7b-96ec-23cd4d68fbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:55.553470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd3589-39f9-4b7b-96ec-23cd4d68fbdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:55.554502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd3589-39f9-4b7b-96ec-23cd4d68fbdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:26:55.554957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd3589-39f9-4b7b-96ec-23cd4d68fbdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:55.556835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd3589-39f9-4b7b-96ec-23cd4d68fbdf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:55.577925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1031/2077] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:26:55.579788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9431e-55bb-4e50-ac3e-fa94288f1919 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:55.590749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9431e-55bb-4e50-ac3e-fa94288f1919 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:55.596574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa89fc-9310-4581-a5c0-1bff2ff91438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:55.597365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa89fc-9310-4581-a5c0-1bff2ff91438 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:55.597952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa89fc-9310-4581-a5c0-1bff2ff91438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:55.598052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa89fc-9310-4581-a5c0-1bff2ff91438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:55.598797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa89fc-9310-4581-a5c0-1bff2ff91438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:55.601126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa89fc-9310-4581-a5c0-1bff2ff91438 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:55.732803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1047/2078] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:55 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:55.734311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd3589-39f9-4b7b-96ec-23cd4d68fbdf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:55.744540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd3589-39f9-4b7b-96ec-23cd4d68fbdf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:55.819228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:55.821085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d639e-7e58-4b61-be07-abed35621f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:55.821899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d639e-7e58-4b61-be07-abed35621f9f admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:55.822365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d639e-7e58-4b61-be07-abed35621f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:55.822636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d639e-7e58-4b61-be07-abed35621f9f admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:55.823522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d639e-7e58-4b61-be07-abed35621f9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:55.826244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d639e-7e58-4b61-be07-abed35621f9f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:55.929771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1032/2079] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:55 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:55.939293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa89fc-9310-4581-a5c0-1bff2ff91438 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:55.954270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa89fc-9310-4581-a5c0-1bff2ff91438 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:55.964584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750c776-158b-47cd-b56f-6f4b0c816f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:55.965209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750c776-158b-47cd-b56f-6f4b0c816f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:55.965642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750c776-158b-47cd-b56f-6f4b0c816f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:55.966359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750c776-158b-47cd-b56f-6f4b0c816f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:55.967053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750c776-158b-47cd-b56f-6f4b0c816f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:55.968771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750c776-158b-47cd-b56f-6f4b0c816f85 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:55.985161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d639e-7e58-4b61-be07-abed35621f9f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:55.990493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1048/2080] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:55.996130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d639e-7e58-4b61-be07-abed35621f9f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:56.001157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e7f48-1b7f-4cb4-8d73-d095179447d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:56.001733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e7f48-1b7f-4cb4-8d73-d095179447d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.002304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e7f48-1b7f-4cb4-8d73-d095179447d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.002391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e7f48-1b7f-4cb4-8d73-d095179447d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.003045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e7f48-1b7f-4cb4-8d73-d095179447d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:56.005538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e7f48-1b7f-4cb4-8d73-d095179447d9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:56.016995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1033/2081] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:56.020975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750c776-158b-47cd-b56f-6f4b0c816f85 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:56.032138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750c776-158b-47cd-b56f-6f4b0c816f85 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:56.037789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3bece-e163-49f6-8d60-9713ee17010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:56.038634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3bece-e163-49f6-8d60-9713ee17010f admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.039376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3bece-e163-49f6-8d60-9713ee17010f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.039376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3bece-e163-49f6-8d60-9713ee17010f admin admin] PATH_INFO: `/v3/projects/22a50df027bf49b39e03f58c22e8a0bc/users/825b13b178c24601b636ffd9f8f92864/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.064000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3bece-e163-49f6-8d60-9713ee17010f admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a50df027bf49b39e03f58c22e8a0bc, user_id=825b13b178c24601b636ffd9f8f92864, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:56.066783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3bece-e163-49f6-8d60-9713ee17010f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:56.168842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1049/2082] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:56 2022] PUT /identity/v3/projects/22a50df027bf49b39e03f58c22e8a0bc/users/825b13b178c24601b636ffd9f8f92864/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:56.179433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec368d-f29b-49fe-8d18-ad4cce80d403 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.180011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec368d-f29b-49fe-8d18-ad4cce80d403 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.180291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec368d-f29b-49fe-8d18-ad4cce80d403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.224764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-35ec368d-f29b-49fe-8d18-ad4cce80d403 None None] MFA Rules not processed for user `825b13b178c24601b636ffd9f8f92864`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:56.226180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-35ec368d-f29b-49fe-8d18-ad4cce80d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:56.385915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1034/2083] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:56.390738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3bece-e163-49f6-8d60-9713ee17010f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:56.407547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:56.454176 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91187d1-f9cb-4371-b011-f36b5e596061 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.454634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91187d1-f9cb-4371-b011-f36b5e596061 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.454939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91187d1-f9cb-4371-b011-f36b5e596061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.491352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a91187d1-f9cb-4371-b011-f36b5e596061 None None] MFA Rules not processed for user `57be68e76fc14d2ca309571e284509bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:56.492588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a91187d1-f9cb-4371-b011-f36b5e596061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:56.500465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3bece-e163-49f6-8d60-9713ee17010f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:56.505530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcdbf1-8bdd-490f-84aa-a8ee6af0c6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:56.506852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcdbf1-8bdd-490f-84aa-a8ee6af0c6ef admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.506852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcdbf1-8bdd-490f-84aa-a8ee6af0c6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.506852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcdbf1-8bdd-490f-84aa-a8ee6af0c6ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.507323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fcdbf1-8bdd-490f-84aa-a8ee6af0c6ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:56.509241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fcdbf1-8bdd-490f-84aa-a8ee6af0c6ef admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:56.573781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1035/2084] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:56.587166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a1f0f-ad00-488f-9d8f-156cd2736f28 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.587166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a1f0f-ad00-488f-9d8f-156cd2736f28 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.587166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a1f0f-ad00-488f-9d8f-156cd2736f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.609564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f12a1f0f-ad00-488f-9d8f-156cd2736f28 None None] MFA Rules not processed for user `57be68e76fc14d2ca309571e284509bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:56.610963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f12a1f0f-ad00-488f-9d8f-156cd2736f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:56.616742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1050/2085] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:26:56 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:56.619824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1036/2086] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:26:56.625139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcdbf1-8bdd-490f-84aa-a8ee6af0c6ef admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:56.641851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a1f0f-ad00-488f-9d8f-156cd2736f28 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:56.645076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f12a1f0f-ad00-488f-9d8f-156cd2736f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:56.737593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcdbf1-8bdd-490f-84aa-a8ee6af0c6ef admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:56.743551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5a8fd-2dae-4465-b663-66844b8b8c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:56.748562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5a8fd-2dae-4465-b663-66844b8b8c63 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.748562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5a8fd-2dae-4465-b663-66844b8b8c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.748562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5a8fd-2dae-4465-b663-66844b8b8c63 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.748562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5a8fd-2dae-4465-b663-66844b8b8c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:56.748982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5a8fd-2dae-4465-b663-66844b8b8c63 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:56.750891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a1f0f-ad00-488f-9d8f-156cd2736f28 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:56.818073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:56.829559 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1051/2087] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:26:56 2022] POST /identity/v3/users => generated 300 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:26:56.831033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5a8fd-2dae-4465-b663-66844b8b8c63 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:56.838903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908fe67-1c36-4102-bd42-2fa8ccdf4059 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:56.839630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908fe67-1c36-4102-bd42-2fa8ccdf4059 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.839985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908fe67-1c36-4102-bd42-2fa8ccdf4059 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.840395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908fe67-1c36-4102-bd42-2fa8ccdf4059 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.842106 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5a8fd-2dae-4465-b663-66844b8b8c63 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:56.848032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbd3e8-6e21-437e-b139-3f45630e7246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:56.848517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbd3e8-6e21-437e-b139-3f45630e7246 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.848933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbd3e8-6e21-437e-b139-3f45630e7246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.849222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbd3e8-6e21-437e-b139-3f45630e7246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.850177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbd3e8-6e21-437e-b139-3f45630e7246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:56.858393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbd3e8-6e21-437e-b139-3f45630e7246 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:56.875382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1052/2088] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:56.876185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbd3e8-6e21-437e-b139-3f45630e7246 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:56.887503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbd3e8-6e21-437e-b139-3f45630e7246 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:56.893819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba83a86-4b93-49fd-aabc-60f8771e9a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:56.894494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba83a86-4b93-49fd-aabc-60f8771e9a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:56.894785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba83a86-4b93-49fd-aabc-60f8771e9a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:56.895070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba83a86-4b93-49fd-aabc-60f8771e9a2d admin admin] PATH_INFO: `/v3/projects/bbcec7f366f94123875bb63cf360e16c/users/a6d788df9b2f441da9da6400516c9f85/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:56.916421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba83a86-4b93-49fd-aabc-60f8771e9a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbcec7f366f94123875bb63cf360e16c, user_id=a6d788df9b2f441da9da6400516c9f85, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:56.920363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba83a86-4b93-49fd-aabc-60f8771e9a2d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:56.922978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908fe67-1c36-4102-bd42-2fa8ccdf4059 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:56.925927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908fe67-1c36-4102-bd42-2fa8ccdf4059 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:56.942151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1037/2089] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:56.999855 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1053/2090] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:56 2022] PUT /identity/v3/projects/bbcec7f366f94123875bb63cf360e16c/users/a6d788df9b2f441da9da6400516c9f85/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:57.005183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908fe67-1c36-4102-bd42-2fa8ccdf4059 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:57.017475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908fe67-1c36-4102-bd42-2fa8ccdf4059 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:57.127531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d5d87-2229-4897-94cc-a475bf657ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:57.128184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d5d87-2229-4897-94cc-a475bf657ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:57.128456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d5d87-2229-4897-94cc-a475bf657ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:57.128728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d5d87-2229-4897-94cc-a475bf657ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:57.129386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d5d87-2229-4897-94cc-a475bf657ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:57.131131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d5d87-2229-4897-94cc-a475bf657ac6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:57.165898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1038/2091] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:57.167272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba83a86-4b93-49fd-aabc-60f8771e9a2d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:57.186187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba83a86-4b93-49fd-aabc-60f8771e9a2d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:57.196549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec58ba-d23c-47ed-a086-9052fe1ed945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:57.198037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec58ba-d23c-47ed-a086-9052fe1ed945 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:57.198793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec58ba-d23c-47ed-a086-9052fe1ed945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:57.199407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec58ba-d23c-47ed-a086-9052fe1ed945 admin admin] PATH_INFO: `/v3/projects/bbcec7f366f94123875bb63cf360e16c/users/a6d788df9b2f441da9da6400516c9f85/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:57.205016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec58ba-d23c-47ed-a086-9052fe1ed945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbcec7f366f94123875bb63cf360e16c, user_id=a6d788df9b2f441da9da6400516c9f85, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:57.208954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec58ba-d23c-47ed-a086-9052fe1ed945 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:57.222349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1054/2092] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:26:57 2022] PUT /identity/v3/projects/bbcec7f366f94123875bb63cf360e16c/users/a6d788df9b2f441da9da6400516c9f85/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:57.235217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182aae2-045d-4997-ba92-cc035e49310c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:57.235512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182aae2-045d-4997-ba92-cc035e49310c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:57.235792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182aae2-045d-4997-ba92-cc035e49310c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:57.288096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1182aae2-045d-4997-ba92-cc035e49310c None None] MFA Rules not processed for user `a6d788df9b2f441da9da6400516c9f85`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:57.290192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1182aae2-045d-4997-ba92-cc035e49310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:57.386874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1039/2093] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:57.398081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec58ba-d23c-47ed-a086-9052fe1ed945 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:57.400183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f4ec58ba-d23c-47ed-a086-9052fe1ed945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:57.465290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec58ba-d23c-47ed-a086-9052fe1ed945 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:57.469611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e25696-fca6-40da-a9f2-64a454514bc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:57.470326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e25696-fca6-40da-a9f2-64a454514bc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:57.470681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e25696-fca6-40da-a9f2-64a454514bc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:57.470961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e25696-fca6-40da-a9f2-64a454514bc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:57.484060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e25696-fca6-40da-a9f2-64a454514bc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:57.485790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e25696-fca6-40da-a9f2-64a454514bc7 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:57.497466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1055/2094] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:57.544192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182aae2-045d-4997-ba92-cc035e49310c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:57.613763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182aae2-045d-4997-ba92-cc035e49310c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:57.619739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:57.620670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:57.621030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:57.621917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] PATH_INFO: `/v3/users/825b13b178c24601b636ffd9f8f92864` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:57.624114 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=825b13b178c24601b636ffd9f8f92864)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:57.626830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:57.858190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '825b13b178c24601b636ffd9f8f92864'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:57.859693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '825b13b178c24601b636ffd9f8f92864'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:57.867078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '825b13b178c24601b636ffd9f8f92864'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:57.969220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aa894-5e44-418c-a199-bd9d102d06cf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:57.969509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aa894-5e44-418c-a199-bd9d102d06cf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:57.969785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aa894-5e44-418c-a199-bd9d102d06cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:57.994109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bb0aa894-5e44-418c-a199-bd9d102d06cf None None] MFA Rules not processed for user `57be68e76fc14d2ca309571e284509bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:57.996195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bb0aa894-5e44-418c-a199-bd9d102d06cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:58.015906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '825b13b178c24601b636ffd9f8f92864'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:58.031470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89dc7cb9-2915-40b6-aff3-c41428025a78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '825b13b178c24601b636ffd9f8f92864'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:58.055217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1040/2095] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:57 2022] DELETE /identity/v3/users/825b13b178c24601b636ffd9f8f92864 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:58.060460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b21eb-f6f0-46f2-8fb6-454f12a24fc8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:58.061033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b21eb-f6f0-46f2-8fb6-454f12a24fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:58.061708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b21eb-f6f0-46f2-8fb6-454f12a24fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:58.100182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1056/2096] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:58.115775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-711b21eb-f6f0-46f2-8fb6-454f12a24fc8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:58.117166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-711b21eb-f6f0-46f2-8fb6-454f12a24fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:58.146410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:58.222290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:58.224231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aa894-5e44-418c-a199-bd9d102d06cf None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:58.225960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bb0aa894-5e44-418c-a199-bd9d102d06cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:58.235917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1041/2097] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:58.251563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aa894-5e44-418c-a199-bd9d102d06cf None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:58.253845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b21eb-f6f0-46f2-8fb6-454f12a24fc8 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:58.256453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-711b21eb-f6f0-46f2-8fb6-454f12a24fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:58.355318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b21eb-f6f0-46f2-8fb6-454f12a24fc8 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:58.359921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c825-84d7-4f2c-958f-1a1de1434e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:58.360766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c825-84d7-4f2c-958f-1a1de1434e8b service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:58.360884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c825-84d7-4f2c-958f-1a1de1434e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:58.361143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c825-84d7-4f2c-958f-1a1de1434e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:58.371543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572c825-84d7-4f2c-958f-1a1de1434e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:58.373300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572c825-84d7-4f2c-958f-1a1de1434e8b service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:58.378936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:58.387426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:58.387955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:58.388251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:58.389545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1042/2098] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:58.391931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:58.500611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:58.502755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:58.515561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1057/2099] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:26:59.012323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c825-84d7-4f2c-958f-1a1de1434e8b service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:59.014244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5572c825-84d7-4f2c-958f-1a1de1434e8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:59.028555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c825-84d7-4f2c-958f-1a1de1434e8b service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:59.033435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:59.034054 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:59.034480 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:59.034765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] PATH_INFO: `/v3/users/e652482a084246e98231bb97bae4064c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:59.043613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e652482a084246e98231bb97bae4064c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:59.047564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:59.181733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:59.184580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:59.204257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571889f-413a-41f0-a1e3-55a7c15469c8 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:59.219925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:59.220512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:59.220856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:59.221453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] PATH_INFO: `/v3/projects/22a50df027bf49b39e03f58c22e8a0bc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:59.222231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a50df027bf49b39e03f58c22e8a0bc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:59.224138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:59.378671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e652482a084246e98231bb97bae4064c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.379504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e652482a084246e98231bb97bae4064c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.380293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e652482a084246e98231bb97bae4064c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.410712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a50df027bf49b39e03f58c22e8a0bc', 'request_id': 'req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.438112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a50df027bf49b39e03f58c22e8a0bc', 'request_id': 'req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.461211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e652482a084246e98231bb97bae4064c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.471186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2630da0a-a65a-4dc4-a977-c7fe4c80f016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e652482a084246e98231bb97bae4064c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.492840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1043/2100] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:59 2022] DELETE /identity/v3/users/e652482a084246e98231bb97bae4064c => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:26:59.498747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ff92e-515d-4a9a-ba98-4a75fffa47d6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:59.500566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ff92e-515d-4a9a-ba98-4a75fffa47d6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:59.500989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ff92e-515d-4a9a-ba98-4a75fffa47d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:59.531723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a88ff92e-515d-4a9a-ba98-4a75fffa47d6 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:59.533809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a88ff92e-515d-4a9a-ba98-4a75fffa47d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:59.570098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a50df027bf49b39e03f58c22e8a0bc', 'request_id': 'req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.590683 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] The token cache is being invalidate because project 22a50df027bf49b39e03f58c22e8a0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:26:59.591229 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.592287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.594798 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1058/2101] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:26:59 2022] DELETE /identity/v3/projects/22a50df027bf49b39e03f58c22e8a0bc => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:26:59.600062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:59.601845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:59.643449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1044/2102] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:59.655829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ff92e-515d-4a9a-ba98-4a75fffa47d6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:59.657907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a88ff92e-515d-4a9a-ba98-4a75fffa47d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:59.706247 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5d5e9-8c6d-47ec-9c71-869fbc3e153a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:59.718403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:59.721388 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:59.721690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:59.721984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] PATH_INFO: `/v3/users/a6d788df9b2f441da9da6400516c9f85` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:59.727216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6d788df9b2f441da9da6400516c9f85)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:59.728215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:59.747667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ff92e-515d-4a9a-ba98-4a75fffa47d6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:59.755302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8d595-d6f0-4653-a1ea-1439c62c7867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:59.757168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8d595-d6f0-4653-a1ea-1439c62c7867 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:59.757620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8d595-d6f0-4653-a1ea-1439c62c7867 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:59.758689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8d595-d6f0-4653-a1ea-1439c62c7867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:59.772565 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8d595-d6f0-4653-a1ea-1439c62c7867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:59.774097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8d595-d6f0-4653-a1ea-1439c62c7867 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:59.786406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1045/2103] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:59.955422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6d788df9b2f441da9da6400516c9f85'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.956124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6d788df9b2f441da9da6400516c9f85'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.956737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6d788df9b2f441da9da6400516c9f85'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.964930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6d788df9b2f441da9da6400516c9f85'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:26:59.979081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6d788df9b2f441da9da6400516c9f85'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:00.082647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1059/2104] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:26:59 2022] DELETE /identity/v3/users/a6d788df9b2f441da9da6400516c9f85 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:00.476120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8d595-d6f0-4653-a1ea-1439c62c7867 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:00.477624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-61a8d595-d6f0-4653-a1ea-1439c62c7867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:00.505297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8d595-d6f0-4653-a1ea-1439c62c7867 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:00.647980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:00.650784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:00.651253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:00.651676 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] PATH_INFO: `/v3/projects/9e9e749165c544e7b311a4f350edf706` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:00.653699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9e749165c544e7b311a4f350edf706)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:00.668479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:00.796825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:00.816933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9e749165c544e7b311a4f350edf706', 'request_id': 'req-47becb35-4245-459d-9546-01afce5d5cb8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:00.822516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9e749165c544e7b311a4f350edf706', 'request_id': 'req-47becb35-4245-459d-9546-01afce5d5cb8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:00.860655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9e749165c544e7b311a4f350edf706', 'request_id': 'req-47becb35-4245-459d-9546-01afce5d5cb8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:00.915207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:00.917905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:00.933411 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d9f3f-7128-456a-aae3-58e8db1cf9ca admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:00.951198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:00.956080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] The token cache is being invalidate because project 9e9e749165c544e7b311a4f350edf706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:00.956445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:00.957466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47becb35-4245-459d-9546-01afce5d5cb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:00.959281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1046/2105] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:00 2022] DELETE /identity/v3/projects/9e9e749165c544e7b311a4f350edf706 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:00.964481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8de04a-6f41-4016-847d-842760d103e7 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:00.964807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8de04a-6f41-4016-847d-842760d103e7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:00.965086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8de04a-6f41-4016-847d-842760d103e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:00.993048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-dc8de04a-6f41-4016-847d-842760d103e7 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:00.995519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dc8de04a-6f41-4016-847d-842760d103e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:01.066353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:01.067507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.067964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.068725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] PATH_INFO: `/v3/projects/bbcec7f366f94123875bb63cf360e16c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.071036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbcec7f366f94123875bb63cf360e16c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:01.078164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:01.129455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1047/2106] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:01.132736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbc2de-cd38-494b-96f4-b12ee3af07a5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.133050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbc2de-cd38-494b-96f4-b12ee3af07a5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.133324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbc2de-cd38-494b-96f4-b12ee3af07a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.196792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f0dbc2de-cd38-494b-96f4-b12ee3af07a5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:01.199485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f0dbc2de-cd38-494b-96f4-b12ee3af07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:01.235426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1048/2107] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:01.239795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbc2de-cd38-494b-96f4-b12ee3af07a5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:01.261285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbc2de-cd38-494b-96f4-b12ee3af07a5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:01.274348 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ed965-e903-42bd-9562-51dd23d4d8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:01.275189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ed965-e903-42bd-9562-51dd23d4d8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.275788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ed965-e903-42bd-9562-51dd23d4d8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.276063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ed965-e903-42bd-9562-51dd23d4d8e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.276779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ed965-e903-42bd-9562-51dd23d4d8e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:01.278358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ed965-e903-42bd-9562-51dd23d4d8e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:01.304922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ed965-e903-42bd-9562-51dd23d4d8e1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:01.322187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbcec7f366f94123875bb63cf360e16c', 'request_id': 'req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:01.342986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1049/2108] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:01.348784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ed965-e903-42bd-9562-51dd23d4d8e1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:01.357620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbcec7f366f94123875bb63cf360e16c', 'request_id': 'req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:01.375050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ed965-e903-42bd-9562-51dd23d4d8e1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:01.455010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbcec7f366f94123875bb63cf360e16c', 'request_id': 'req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:01.473979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] The token cache is being invalidate because project bbcec7f366f94123875bb63cf360e16c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:01.474201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:01.478330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-591b6a72-348a-4f7a-beed-0ebf9e3ceb94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:01.481135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1060/2109] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:00 2022] DELETE /identity/v3/projects/bbcec7f366f94123875bb63cf360e16c => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:01.488555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c8fe9-5740-49b8-9bac-585e3d521020 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.488852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c8fe9-5740-49b8-9bac-585e3d521020 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.489133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c8fe9-5740-49b8-9bac-585e3d521020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.539677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e18b6-ec35-47f6-ae60-c85efdb355a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:01.540179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-703c8fe9-5740-49b8-9bac-585e3d521020 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:01.540954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-703c8fe9-5740-49b8-9bac-585e3d521020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:01.541214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e18b6-ec35-47f6-ae60-c85efdb355a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.541635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e18b6-ec35-47f6-ae60-c85efdb355a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.542398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e18b6-ec35-47f6-ae60-c85efdb355a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.544129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e18b6-ec35-47f6-ae60-c85efdb355a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:01.549411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e18b6-ec35-47f6-ae60-c85efdb355a3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:01.558286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1061/2110] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:01.559730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cd70f-0b83-4980-8575-4c3794da5d89 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.560022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cd70f-0b83-4980-8575-4c3794da5d89 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.560300 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cd70f-0b83-4980-8575-4c3794da5d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.591326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f84cd70f-0b83-4980-8575-4c3794da5d89 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:01.592562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f84cd70f-0b83-4980-8575-4c3794da5d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:01.604473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1062/2111] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:01.609925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cd70f-0b83-4980-8575-4c3794da5d89 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:01.629810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cd70f-0b83-4980-8575-4c3794da5d89 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:01.635024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a98f57-8928-4c03-8ee5-7ace870d88f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:01.635719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a98f57-8928-4c03-8ee5-7ace870d88f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.636005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a98f57-8928-4c03-8ee5-7ace870d88f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.636285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a98f57-8928-4c03-8ee5-7ace870d88f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.637542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a98f57-8928-4c03-8ee5-7ace870d88f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:01.637851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a98f57-8928-4c03-8ee5-7ace870d88f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:01.639514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a98f57-8928-4c03-8ee5-7ace870d88f7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:01.652818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1063/2112] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:27:01.656446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a98f57-8928-4c03-8ee5-7ace870d88f7 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:01.667713 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a98f57-8928-4c03-8ee5-7ace870d88f7 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:01.675400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40789e7-0679-4a1b-8ac1-146dce5499a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:01.682076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40789e7-0679-4a1b-8ac1-146dce5499a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.682076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40789e7-0679-4a1b-8ac1-146dce5499a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.682076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40789e7-0679-4a1b-8ac1-146dce5499a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.682076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40789e7-0679-4a1b-8ac1-146dce5499a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:01.685282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40789e7-0679-4a1b-8ac1-146dce5499a3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:01.739383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1050/2113] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:01 2022] POST /identity/v3/projects => generated 373 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:01.743858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e18b6-ec35-47f6-ae60-c85efdb355a3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:01.748897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-760e18b6-ec35-47f6-ae60-c85efdb355a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:01.835062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1064/2114] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:01 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:27:01.839317 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40789e7-0679-4a1b-8ac1-146dce5499a3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:01.849330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e18b6-ec35-47f6-ae60-c85efdb355a3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:01.930486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40789e7-0679-4a1b-8ac1-146dce5499a3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:01.937331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83ec4f-b46b-4e2e-9b49-62efcfd8f8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:01.938205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83ec4f-b46b-4e2e-9b49-62efcfd8f8be admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.938636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83ec4f-b46b-4e2e-9b49-62efcfd8f8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.939041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83ec4f-b46b-4e2e-9b49-62efcfd8f8be admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.939848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83ec4f-b46b-4e2e-9b49-62efcfd8f8be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:01.941030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb242e0c-52a6-4d81-80ed-043cc8d897e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:01.942049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83ec4f-b46b-4e2e-9b49-62efcfd8f8be admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:01.942189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb242e0c-52a6-4d81-80ed-043cc8d897e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:01.942574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb242e0c-52a6-4d81-80ed-043cc8d897e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:01.943257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb242e0c-52a6-4d81-80ed-043cc8d897e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:01.944532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb242e0c-52a6-4d81-80ed-043cc8d897e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:01.947211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb242e0c-52a6-4d81-80ed-043cc8d897e1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:02.081448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1051/2115] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:01 2022] POST /identity/v3/users => generated 287 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:27:02.084067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1065/2116] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:01 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:27:02.086959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb242e0c-52a6-4d81-80ed-043cc8d897e1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:02.088102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83ec4f-b46b-4e2e-9b49-62efcfd8f8be admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:02.099447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb242e0c-52a6-4d81-80ed-043cc8d897e1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:02.105199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83ec4f-b46b-4e2e-9b49-62efcfd8f8be admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:02.106986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d14ec-f067-4191-8a66-a3555c2049c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:02.109952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d14ec-f067-4191-8a66-a3555c2049c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.109952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d14ec-f067-4191-8a66-a3555c2049c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.109952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d14ec-f067-4191-8a66-a3555c2049c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.111188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d14ec-f067-4191-8a66-a3555c2049c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:02.113559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d14ec-f067-4191-8a66-a3555c2049c3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:02.122367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7383-e2b3-4eb0-bcd8-0b8f586b5f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:02.123585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7383-e2b3-4eb0-bcd8-0b8f586b5f9f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.123876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7383-e2b3-4eb0-bcd8-0b8f586b5f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.124507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7383-e2b3-4eb0-bcd8-0b8f586b5f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.125253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab7383-e2b3-4eb0-bcd8-0b8f586b5f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:02.126519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1052/2117] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:02.127063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab7383-e2b3-4eb0-bcd8-0b8f586b5f9f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:02.130893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d14ec-f067-4191-8a66-a3555c2049c3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:02.142754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1066/2118] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:02.149561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7383-e2b3-4eb0-bcd8-0b8f586b5f9f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:02.162443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7383-e2b3-4eb0-bcd8-0b8f586b5f9f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:02.167377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d14ec-f067-4191-8a66-a3555c2049c3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:02.169506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb472df-f8e1-431e-87d9-d8b83aca7b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:02.170661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb472df-f8e1-431e-87d9-d8b83aca7b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.171119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb472df-f8e1-431e-87d9-d8b83aca7b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.171652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb472df-f8e1-431e-87d9-d8b83aca7b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.172561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb472df-f8e1-431e-87d9-d8b83aca7b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:02.174626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb472df-f8e1-431e-87d9-d8b83aca7b1b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:02.184772 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c15c9-20f6-4d8e-849b-31a0f0f32185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:02.185413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c15c9-20f6-4d8e-849b-31a0f0f32185 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.186406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c15c9-20f6-4d8e-849b-31a0f0f32185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.186689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c15c9-20f6-4d8e-849b-31a0f0f32185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.187495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c15c9-20f6-4d8e-849b-31a0f0f32185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:02.191226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c15c9-20f6-4d8e-849b-31a0f0f32185 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:02.192179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1067/2119] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:02.193445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb472df-f8e1-431e-87d9-d8b83aca7b1b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:02.204327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1053/2120] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:02.208993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c15c9-20f6-4d8e-849b-31a0f0f32185 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:02.209570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb472df-f8e1-431e-87d9-d8b83aca7b1b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:02.215719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e00fe-beed-4200-8b9e-665d8a222360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:02.216566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e00fe-beed-4200-8b9e-665d8a222360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.219852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e00fe-beed-4200-8b9e-665d8a222360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.219852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e00fe-beed-4200-8b9e-665d8a222360 admin admin] PATH_INFO: `/v3/projects/5f631a1b24314036ad3844cf1a083a52/users/5ba62f324b04453181b82917b303f765/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.230401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c15c9-20f6-4d8e-849b-31a0f0f32185 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:02.237182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e00fe-beed-4200-8b9e-665d8a222360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f631a1b24314036ad3844cf1a083a52, user_id=5ba62f324b04453181b82917b303f765, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:02.237826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f675b2-19e6-4375-b06a-fa0d7144b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:02.238722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f675b2-19e6-4375-b06a-fa0d7144b457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.239007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f675b2-19e6-4375-b06a-fa0d7144b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.240747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f675b2-19e6-4375-b06a-fa0d7144b457 admin admin] PATH_INFO: `/v3/projects/a1b8e9dbce62475eaa97dbad91fcd910/users/8a512d59d7b4477b88f7bb9532141d06/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.241381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e00fe-beed-4200-8b9e-665d8a222360 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:02.271941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f675b2-19e6-4375-b06a-fa0d7144b457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b8e9dbce62475eaa97dbad91fcd910, user_id=8a512d59d7b4477b88f7bb9532141d06, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:02.276224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f675b2-19e6-4375-b06a-fa0d7144b457 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:02.336296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:02.344254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1068/2121] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:02 2022] PUT /identity/v3/projects/5f631a1b24314036ad3844cf1a083a52/users/5ba62f324b04453181b82917b303f765/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:02.356156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f9f0-2b28-475e-a442-395370c62dae None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.356461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f9f0-2b28-475e-a442-395370c62dae None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.356691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f9f0-2b28-475e-a442-395370c62dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.414822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1054/2122] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:02 2022] PUT /identity/v3/projects/a1b8e9dbce62475eaa97dbad91fcd910/users/8a512d59d7b4477b88f7bb9532141d06/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:02.421456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:02.423578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fca87-33fd-454d-97d1-c29fd555ed6e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.424073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fca87-33fd-454d-97d1-c29fd555ed6e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.424781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fca87-33fd-454d-97d1-c29fd555ed6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.427598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7c07f9f0-2b28-475e-a442-395370c62dae None None] MFA Rules not processed for user `5ba62f324b04453181b82917b303f765`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:02.429080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7c07f9f0-2b28-475e-a442-395370c62dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:02.452422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4a9fca87-33fd-454d-97d1-c29fd555ed6e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:02.453679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4a9fca87-33fd-454d-97d1-c29fd555ed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:02.553617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1069/2123] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:02.558189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07f9f0-2b28-475e-a442-395370c62dae None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:02.561838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1055/2124] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:02.569894 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fca87-33fd-454d-97d1-c29fd555ed6e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:02.570354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4a9fca87-33fd-454d-97d1-c29fd555ed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:02.571489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07f9f0-2b28-475e-a442-395370c62dae None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:02.577476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0230e-e89a-4251-905b-ff983f64ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:02.578100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0230e-e89a-4251-905b-ff983f64ea79 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.578383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0230e-e89a-4251-905b-ff983f64ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.578659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0230e-e89a-4251-905b-ff983f64ea79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.581617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0230e-e89a-4251-905b-ff983f64ea79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:02.581617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0230e-e89a-4251-905b-ff983f64ea79 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:02.646116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fca87-33fd-454d-97d1-c29fd555ed6e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:02.651758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74874dc6-2c8c-423f-9405-db4206be8e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:02.652415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74874dc6-2c8c-423f-9405-db4206be8e35 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.652695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74874dc6-2c8c-423f-9405-db4206be8e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.652961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74874dc6-2c8c-423f-9405-db4206be8e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.663133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74874dc6-2c8c-423f-9405-db4206be8e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:02.664834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74874dc6-2c8c-423f-9405-db4206be8e35 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:02.677146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1056/2125] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:27:02.717203 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1070/2126] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:02 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:27:02.721884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74874dc6-2c8c-423f-9405-db4206be8e35 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:02.732182 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74874dc6-2c8c-423f-9405-db4206be8e35 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:02.819140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579501d-e745-4a9b-9653-51ea7200eb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:02.820132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579501d-e745-4a9b-9653-51ea7200eb5e admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:02.820697 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579501d-e745-4a9b-9653-51ea7200eb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:02.821700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579501d-e745-4a9b-9653-51ea7200eb5e admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:02.822986 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579501d-e745-4a9b-9653-51ea7200eb5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:02.826773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579501d-e745-4a9b-9653-51ea7200eb5e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:03.043928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1057/2127] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:02 2022] POST /identity/v3/users => generated 297 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:03.048565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0230e-e89a-4251-905b-ff983f64ea79 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:03.061311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0230e-e89a-4251-905b-ff983f64ea79 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:03.067710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f9597-e72c-4d27-870f-604e075d39b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:03.068778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f9597-e72c-4d27-870f-604e075d39b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:03.069069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f9597-e72c-4d27-870f-604e075d39b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:03.069345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f9597-e72c-4d27-870f-604e075d39b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:03.070119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f9597-e72c-4d27-870f-604e075d39b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:03.072226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f9597-e72c-4d27-870f-604e075d39b6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:03.088462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579501d-e745-4a9b-9653-51ea7200eb5e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:03.090289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1071/2128] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:03.103470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579501d-e745-4a9b-9653-51ea7200eb5e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:03.114652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ea9d-3273-400d-a7b1-c29ff951c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:03.115771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ea9d-3273-400d-a7b1-c29ff951c9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:03.116256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ea9d-3273-400d-a7b1-c29ff951c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:03.116730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ea9d-3273-400d-a7b1-c29ff951c9d7 admin admin] PATH_INFO: `/v3/projects/e01c531806484c6aac10a6e318e060c2/users/69fa93e5f4564fdbb1492a120c464774/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:03.134424 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039ea9d-3273-400d-a7b1-c29ff951c9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01c531806484c6aac10a6e318e060c2, user_id=69fa93e5f4564fdbb1492a120c464774, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:03.137635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039ea9d-3273-400d-a7b1-c29ff951c9d7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:03.157520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:03.195949 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f9597-e72c-4d27-870f-604e075d39b6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:03.197843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-070f9597-e72c-4d27-870f-604e075d39b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:03.213484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f9597-e72c-4d27-870f-604e075d39b6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:03.228360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:03.228860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:03.229148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:03.229421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] PATH_INFO: `/v3/users/6fa21aafc8704e0fb8321363598a040b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:03.232690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa21aafc8704e0fb8321363598a040b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:03.234984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:03.280391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1058/2129] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:03 2022] PUT /identity/v3/projects/e01c531806484c6aac10a6e318e060c2/users/69fa93e5f4564fdbb1492a120c464774/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:03.286623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ea9d-3273-400d-a7b1-c29ff951c9d7 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:03.430186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ea9d-3273-400d-a7b1-c29ff951c9d7 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:03.441222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ba297-bdac-4579-be52-74d975c763f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:03.441992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ba297-bdac-4579-be52-74d975c763f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:03.442314 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ba297-bdac-4579-be52-74d975c763f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:03.442634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ba297-bdac-4579-be52-74d975c763f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:03.443250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ba297-bdac-4579-be52-74d975c763f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:03.444874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ba297-bdac-4579-be52-74d975c763f3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:03.457916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1059/2130] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:27:03.466674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ba297-bdac-4579-be52-74d975c763f3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:03.477692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ba297-bdac-4579-be52-74d975c763f3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:03.485765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c68dc-19d2-488a-8acc-a6ba67cde2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:03.486754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c68dc-19d2-488a-8acc-a6ba67cde2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:03.487047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c68dc-19d2-488a-8acc-a6ba67cde2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:03.487324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c68dc-19d2-488a-8acc-a6ba67cde2ba admin admin] PATH_INFO: `/v3/projects/e01c531806484c6aac10a6e318e060c2/users/69fa93e5f4564fdbb1492a120c464774/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:03.489921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c68dc-19d2-488a-8acc-a6ba67cde2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01c531806484c6aac10a6e318e060c2, user_id=69fa93e5f4564fdbb1492a120c464774, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:03.492694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c68dc-19d2-488a-8acc-a6ba67cde2ba admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:03.506099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1060/2131] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:03 2022] PUT /identity/v3/projects/e01c531806484c6aac10a6e318e060c2/users/69fa93e5f4564fdbb1492a120c464774/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:03.520609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759eac4-9034-432e-8c9e-6cec1bd14a69 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:03.520609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759eac4-9034-432e-8c9e-6cec1bd14a69 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:03.520609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759eac4-9034-432e-8c9e-6cec1bd14a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:03.563875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa21aafc8704e0fb8321363598a040b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:03.564562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa21aafc8704e0fb8321363598a040b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:03.565169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa21aafc8704e0fb8321363598a040b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:03.573327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa21aafc8704e0fb8321363598a040b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:03.574081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a759eac4-9034-432e-8c9e-6cec1bd14a69 None None] MFA Rules not processed for user `69fa93e5f4564fdbb1492a120c464774`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:03.575690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a759eac4-9034-432e-8c9e-6cec1bd14a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:03.589224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7fd8e82d-d0a3-4fa3-b016-d32251e25740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa21aafc8704e0fb8321363598a040b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:03.677216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1061/2132] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:03.685202 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759eac4-9034-432e-8c9e-6cec1bd14a69 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:03.686793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a759eac4-9034-432e-8c9e-6cec1bd14a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:03.702456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1072/2133] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:03 2022] DELETE /identity/v3/users/6fa21aafc8704e0fb8321363598a040b => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:03.706663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bbbd9-cace-4b29-aed8-30b3f93b05a8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:03.707455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bbbd9-cace-4b29-aed8-30b3f93b05a8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:03.707455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bbbd9-cace-4b29-aed8-30b3f93b05a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:03.746426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0c6bbbd9-cace-4b29-aed8-30b3f93b05a8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:03.747910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0c6bbbd9-cace-4b29-aed8-30b3f93b05a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:03.796660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759eac4-9034-432e-8c9e-6cec1bd14a69 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:03.862487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1073/2134] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:03.878050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bbbd9-cace-4b29-aed8-30b3f93b05a8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:03.880009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0c6bbbd9-cace-4b29-aed8-30b3f93b05a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:03.884685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f97d0b-f932-4b85-8245-5d0ded2b6fa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:03.886391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f97d0b-f932-4b85-8245-5d0ded2b6fa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:03.889770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f97d0b-f932-4b85-8245-5d0ded2b6fa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:03.895519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f97d0b-f932-4b85-8245-5d0ded2b6fa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:03.988708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bbbd9-cace-4b29-aed8-30b3f93b05a8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:03.994097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79f7a4-f8c2-43e0-a65e-f2beabe33fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:03.994980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79f7a4-f8c2-43e0-a65e-f2beabe33fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:03.995647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79f7a4-f8c2-43e0-a65e-f2beabe33fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:03.996634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79f7a4-f8c2-43e0-a65e-f2beabe33fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:03.997906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f97d0b-f932-4b85-8245-5d0ded2b6fa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:03.999652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f97d0b-f932-4b85-8245-5d0ded2b6fa2 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:04.008196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79f7a4-f8c2-43e0-a65e-f2beabe33fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:04.009813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79f7a4-f8c2-43e0-a65e-f2beabe33fa7 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:04.013261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1062/2135] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:04.033988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1074/2136] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:27:04.267665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f97d0b-f932-4b85-8245-5d0ded2b6fa2 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:04.278241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f97d0b-f932-4b85-8245-5d0ded2b6fa2 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:04.293821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8a6f4-7568-43fa-a68b-0eb0626c23d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:04.294599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8a6f4-7568-43fa-a68b-0eb0626c23d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:04.297751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8a6f4-7568-43fa-a68b-0eb0626c23d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:04.297751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8a6f4-7568-43fa-a68b-0eb0626c23d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:04.300386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b9e8a6f4-7568-43fa-a68b-0eb0626c23d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:04.396783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:04.412141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8a6f4-7568-43fa-a68b-0eb0626c23d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:04.413890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8a6f4-7568-43fa-a68b-0eb0626c23d5 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:04.424862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1063/2137] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:04.592725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:04.710260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79f7a4-f8c2-43e0-a65e-f2beabe33fa7 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:04.712025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7c79f7a4-f8c2-43e0-a65e-f2beabe33fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:04.728540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79f7a4-f8c2-43e0-a65e-f2beabe33fa7 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:04.735281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:04.736136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:04.736487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:04.736984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] PATH_INFO: `/v3/projects/85eeb65090dc48dcb1bead1b1b1422e0` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:04.738634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85eeb65090dc48dcb1bead1b1b1422e0)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:04.741486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:04.948104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85eeb65090dc48dcb1bead1b1b1422e0', 'request_id': 'req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:04.965274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85eeb65090dc48dcb1bead1b1b1422e0', 'request_id': 'req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.124067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85eeb65090dc48dcb1bead1b1b1422e0', 'request_id': 'req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.136765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] The token cache is being invalidate because project 85eeb65090dc48dcb1bead1b1b1422e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:05.137055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.137737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.139393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1075/2138] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:04 2022] DELETE /identity/v3/projects/85eeb65090dc48dcb1bead1b1b1422e0 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:05.143356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8a6f4-7568-43fa-a68b-0eb0626c23d5 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:05.154628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8a6f4-7568-43fa-a68b-0eb0626c23d5 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:05.205362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:05.211275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:05.299182 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:05.299944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:05.300398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:05.300864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] PATH_INFO: `/v3/users/42fc023146ed4a669d1daaed19330e07` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:05.302456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=42fc023146ed4a669d1daaed19330e07)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:05.304831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:05.365344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833b3d6-5268-4d06-bc9d-b9efee7d9ed3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:05.371730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:05.372677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:05.373113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:05.373540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] PATH_INFO: `/v3/users/5ba62f324b04453181b82917b303f765` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:05.375084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba62f324b04453181b82917b303f765)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:05.377315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:05.582336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42fc023146ed4a669d1daaed19330e07'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.583078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42fc023146ed4a669d1daaed19330e07'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.583689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42fc023146ed4a669d1daaed19330e07'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.669209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba62f324b04453181b82917b303f765'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.669950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba62f324b04453181b82917b303f765'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.670568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba62f324b04453181b82917b303f765'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.676851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42fc023146ed4a669d1daaed19330e07'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.678957 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba62f324b04453181b82917b303f765'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.684332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-053dd86a-0d40-430a-ae30-a56194c37bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42fc023146ed4a669d1daaed19330e07'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.691160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba62f324b04453181b82917b303f765'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:05.702396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1064/2139] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:05 2022] DELETE /identity/v3/users/42fc023146ed4a669d1daaed19330e07 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:05.704433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afcf0e-482a-4f13-93d6-30ca36d87935 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:05.704793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afcf0e-482a-4f13-93d6-30ca36d87935 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:05.705135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afcf0e-482a-4f13-93d6-30ca36d87935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:05.733798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b5afcf0e-482a-4f13-93d6-30ca36d87935 None None] MFA Rules not processed for user `8a512d59d7b4477b88f7bb9532141d06`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:05.735098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b5afcf0e-482a-4f13-93d6-30ca36d87935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:05.793127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1076/2140] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:05 2022] DELETE /identity/v3/users/5ba62f324b04453181b82917b303f765 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:05.794409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:05.796393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:05.804728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1065/2141] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:05.806437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0581be1-8603-470a-9cd2-036f0e2c1d93 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:05.806721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0581be1-8603-470a-9cd2-036f0e2c1d93 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:05.806995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0581be1-8603-470a-9cd2-036f0e2c1d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:05.828360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e0581be1-8603-470a-9cd2-036f0e2c1d93 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:05.829573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e0581be1-8603-470a-9cd2-036f0e2c1d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:05.864520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4a8d6-2819-4848-a96b-bd3457f5e440 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:05.869796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:05.870469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:05.870797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:05.871073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] PATH_INFO: `/v3/users/d63295214c5f415eac699fbe4f0ed3b7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:05.872253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d63295214c5f415eac699fbe4f0ed3b7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:05.874239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:05.889941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1066/2142] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:05.893201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0581be1-8603-470a-9cd2-036f0e2c1d93 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:05.893201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e0581be1-8603-470a-9cd2-036f0e2c1d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:05.950506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0581be1-8603-470a-9cd2-036f0e2c1d93 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:05.954363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f005-7539-49fa-810b-bf3501ce79a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:05.955240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f005-7539-49fa-810b-bf3501ce79a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:05.955240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f005-7539-49fa-810b-bf3501ce79a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:05.955501 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f005-7539-49fa-810b-bf3501ce79a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:06.017302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f005-7539-49fa-810b-bf3501ce79a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:06.018780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f005-7539-49fa-810b-bf3501ce79a5 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:06.028867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1067/2143] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:06.029720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f005-7539-49fa-810b-bf3501ce79a5 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:06.031075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-eef9f005-7539-49fa-810b-bf3501ce79a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:06.089364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f005-7539-49fa-810b-bf3501ce79a5 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:06.093601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90b017-31e4-4aca-ad67-3cf6ef43d9d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:06.094235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b017-31e4-4aca-ad67-3cf6ef43d9d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:06.094602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b017-31e4-4aca-ad67-3cf6ef43d9d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:06.094782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b017-31e4-4aca-ad67-3cf6ef43d9d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:06.104786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90b017-31e4-4aca-ad67-3cf6ef43d9d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:06.107311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90b017-31e4-4aca-ad67-3cf6ef43d9d5 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:06.119824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1068/2144] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:06.257358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd63295214c5f415eac699fbe4f0ed3b7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.258048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd63295214c5f415eac699fbe4f0ed3b7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.258904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd63295214c5f415eac699fbe4f0ed3b7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.265784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd63295214c5f415eac699fbe4f0ed3b7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.278339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-57e1ff2c-7574-4d6a-a718-c218003b5a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd63295214c5f415eac699fbe4f0ed3b7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.297482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90b017-31e4-4aca-ad67-3cf6ef43d9d5 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:06.299065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8d90b017-31e4-4aca-ad67-3cf6ef43d9d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:06.313363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90b017-31e4-4aca-ad67-3cf6ef43d9d5 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:06.330472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:06.331755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:06.332109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:06.332304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] PATH_INFO: `/v3/projects/0ed5ae1d0d5c4522a5c4921539f801fa` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:06.333770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ed5ae1d0d5c4522a5c4921539f801fa)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:06.335774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:06.381837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1077/2145] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:05 2022] DELETE /identity/v3/users/d63295214c5f415eac699fbe4f0ed3b7 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:06.386937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6238c-81f5-4f89-ad8f-6256e1690c14 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:06.387333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6238c-81f5-4f89-ad8f-6256e1690c14 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:06.387819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6238c-81f5-4f89-ad8f-6256e1690c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:06.407528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-75f6238c-81f5-4f89-ad8f-6256e1690c14 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:06.408809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-75f6238c-81f5-4f89-ad8f-6256e1690c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:06.483592 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1078/2146] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:06.484625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6238c-81f5-4f89-ad8f-6256e1690c14 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:06.486460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-75f6238c-81f5-4f89-ad8f-6256e1690c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:06.499956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6238c-81f5-4f89-ad8f-6256e1690c14 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:06.504679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ed5ae1d0d5c4522a5c4921539f801fa', 'request_id': 'req-68c37933-e767-47fd-8124-0cd344079cc9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.510897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ed5ae1d0d5c4522a5c4921539f801fa', 'request_id': 'req-68c37933-e767-47fd-8124-0cd344079cc9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.521725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ed5ae1d0d5c4522a5c4921539f801fa', 'request_id': 'req-68c37933-e767-47fd-8124-0cd344079cc9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.563710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:06.564421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:06.564710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:06.565000 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] PATH_INFO: `/v3/users/70e1b5a6031a4de4b53c043f21ca0738` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:06.566548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70e1b5a6031a4de4b53c043f21ca0738)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:06.568600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:06.605340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] The token cache is being invalidate because project 0ed5ae1d0d5c4522a5c4921539f801fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:06.605653 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.606517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.609526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1069/2147] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:06 2022] DELETE /identity/v3/projects/0ed5ae1d0d5c4522a5c4921539f801fa => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:06.610451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:06.611829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:06.694480 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c37933-e767-47fd-8124-0cd344079cc9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:06.698634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95528bfb-9b4e-4661-8281-d734d53dd0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:06.699929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95528bfb-9b4e-4661-8281-d734d53dd0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:06.700281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95528bfb-9b4e-4661-8281-d734d53dd0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:06.703993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95528bfb-9b4e-4661-8281-d734d53dd0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:06.707438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-95528bfb-9b4e-4661-8281-d734d53dd0a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:06.728039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95528bfb-9b4e-4661-8281-d734d53dd0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:06.731887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95528bfb-9b4e-4661-8281-d734d53dd0a6 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:06.745381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1070/2148] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:06.747159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc751-91c9-445f-9236-b029aa828489 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:06.747532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc751-91c9-445f-9236-b029aa828489 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:06.747872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc751-91c9-445f-9236-b029aa828489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:06.768701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-057cc751-91c9-445f-9236-b029aa828489 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:06.770622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-057cc751-91c9-445f-9236-b029aa828489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:06.776725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:06.796017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1071/2149] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:06.796974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cc751-91c9-445f-9236-b029aa828489 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:06.797451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70e1b5a6031a4de4b53c043f21ca0738'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.798586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-057cc751-91c9-445f-9236-b029aa828489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:06.799088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70e1b5a6031a4de4b53c043f21ca0738'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.799737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70e1b5a6031a4de4b53c043f21ca0738'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.811210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70e1b5a6031a4de4b53c043f21ca0738'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.817806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cc751-91c9-445f-9236-b029aa828489 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:06.818959 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:06.820226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:06.820606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:06.820938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] PATH_INFO: `/v3/projects/5f631a1b24314036ad3844cf1a083a52` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:06.822160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f631a1b24314036ad3844cf1a083a52)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:06.824793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:06.827295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a3eb815-c4a8-46f4-a508-9546cf96edf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70e1b5a6031a4de4b53c043f21ca0738'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:06.945973 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1079/2150] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:06 2022] DELETE /identity/v3/users/70e1b5a6031a4de4b53c043f21ca0738 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:06.947267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e638d4-1eed-4ad1-80da-0b9797481215 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:06.947554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e638d4-1eed-4ad1-80da-0b9797481215 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:06.948033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e638d4-1eed-4ad1-80da-0b9797481215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:06.968178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a7e638d4-1eed-4ad1-80da-0b9797481215 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:06.969391 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a7e638d4-1eed-4ad1-80da-0b9797481215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.007490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f631a1b24314036ad3844cf1a083a52', 'request_id': 'req-674728f4-4616-4ee9-b416-4beb2feedc7a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.014762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f631a1b24314036ad3844cf1a083a52', 'request_id': 'req-674728f4-4616-4ee9-b416-4beb2feedc7a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.021744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f631a1b24314036ad3844cf1a083a52', 'request_id': 'req-674728f4-4616-4ee9-b416-4beb2feedc7a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.037492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1080/2151] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:07.038980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f58aa2-7cc6-4865-a672-873c10ef17c3 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.039281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f58aa2-7cc6-4865-a672-873c10ef17c3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.039859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f58aa2-7cc6-4865-a672-873c10ef17c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.070795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-97f58aa2-7cc6-4865-a672-873c10ef17c3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:07.070795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-97f58aa2-7cc6-4865-a672-873c10ef17c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.124100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1081/2152] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:07.125197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f58aa2-7cc6-4865-a672-873c10ef17c3 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.134183 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:07.136954 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] The token cache is being invalidate because project 5f631a1b24314036ad3844cf1a083a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:07.137428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.138169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.139030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f58aa2-7cc6-4865-a672-873c10ef17c3 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:07.139844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1072/2153] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:06 2022] DELETE /identity/v3/projects/5f631a1b24314036ad3844cf1a083a52 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:07.140713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.142108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.145115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:07.145678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.146130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.147876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.147876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:07.147876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:07.148954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:07.162169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1082/2154] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:07.162397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.166323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.180976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833973b3-3377-403d-97c0-c0ce54625f2b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:07.186406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:07.187034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.187311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.196714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] PATH_INFO: `/v3/users/69fa93e5f4564fdbb1492a120c464774` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.196714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69fa93e5f4564fdbb1492a120c464774)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:07.196714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:07.206328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674728f4-4616-4ee9-b416-4beb2feedc7a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:07.210116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:07.210710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.210983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.211259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.213021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.226859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:07.227974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:07.238154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1073/2155] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:07.238901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.240280 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.254511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8aa5a-2e5b-4051-9c05-5022f0c5148c service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:07.259625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930995c-774c-474c-a6fa-374554ac11f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:07.260562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930995c-774c-474c-a6fa-374554ac11f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.260891 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930995c-774c-474c-a6fa-374554ac11f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.261166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930995c-774c-474c-a6fa-374554ac11f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.261846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930995c-774c-474c-a6fa-374554ac11f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:07.264013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930995c-774c-474c-a6fa-374554ac11f9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:07.340320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1074/2156] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:07 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:07.340320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930995c-774c-474c-a6fa-374554ac11f9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.340320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5930995c-774c-474c-a6fa-374554ac11f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.378351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69fa93e5f4564fdbb1492a120c464774'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.378948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69fa93e5f4564fdbb1492a120c464774'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.379694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69fa93e5f4564fdbb1492a120c464774'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.388194 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69fa93e5f4564fdbb1492a120c464774'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.407528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69fa93e5f4564fdbb1492a120c464774'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.410263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930995c-774c-474c-a6fa-374554ac11f9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:07.418530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:07.419181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.419459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.419787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] PATH_INFO: `/v3/projects/686d419304d44ecdaab67cf2aa926e94` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.420953 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=686d419304d44ecdaab67cf2aa926e94)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:07.422956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:07.460160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1083/2157] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:07 2022] DELETE /identity/v3/users/69fa93e5f4564fdbb1492a120c464774 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:07.460160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.460160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.544970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '686d419304d44ecdaab67cf2aa926e94', 'request_id': 'req-9c59530a-b433-42ec-bdfc-a0681cc12e00'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.552546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '686d419304d44ecdaab67cf2aa926e94', 'request_id': 'req-9c59530a-b433-42ec-bdfc-a0681cc12e00'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.559765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '686d419304d44ecdaab67cf2aa926e94', 'request_id': 'req-9c59530a-b433-42ec-bdfc-a0681cc12e00'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.568455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8da364-2d6c-498e-b341-07ffcc8d2197 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:07.628605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d0585-ce62-4657-8422-678078a9aced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:07.629140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d0585-ce62-4657-8422-678078a9aced admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.629423 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d0585-ce62-4657-8422-678078a9aced admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.629700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d0585-ce62-4657-8422-678078a9aced admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.630365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d0585-ce62-4657-8422-678078a9aced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:07.632316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d0585-ce62-4657-8422-678078a9aced admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:07.678572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] The token cache is being invalidate because project 686d419304d44ecdaab67cf2aa926e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:07.678859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.684512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.684512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1075/2158] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:07 2022] DELETE /identity/v3/projects/686d419304d44ecdaab67cf2aa926e94 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:07.684512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.684512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.703391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59530a-b433-42ec-bdfc-a0681cc12e00 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:07.710830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:07.710830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.710830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.710830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] PATH_INFO: `/v3/projects/24305a12daa54c6c94ec0fdd234ab400` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.712243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24305a12daa54c6c94ec0fdd234ab400)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:07.714812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:07.782706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1084/2159] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:07 2022] POST /identity/v3/users => generated 295 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:07.783976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40429765-599a-478f-8ccb-333eca06f765 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.784263 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40429765-599a-478f-8ccb-333eca06f765 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.784540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40429765-599a-478f-8ccb-333eca06f765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.822550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-40429765-599a-478f-8ccb-333eca06f765 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:07.825440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-40429765-599a-478f-8ccb-333eca06f765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.835995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1085/2160] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:07.837015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40429765-599a-478f-8ccb-333eca06f765 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.838332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-40429765-599a-478f-8ccb-333eca06f765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.851660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40429765-599a-478f-8ccb-333eca06f765 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:07.861660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24305a12daa54c6c94ec0fdd234ab400', 'request_id': 'req-9c659c82-bd40-4ac8-9fff-255aab51f8b7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.867708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24305a12daa54c6c94ec0fdd234ab400', 'request_id': 'req-9c659c82-bd40-4ac8-9fff-255aab51f8b7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.876131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24305a12daa54c6c94ec0fdd234ab400', 'request_id': 'req-9c659c82-bd40-4ac8-9fff-255aab51f8b7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.916403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b21e5-a287-4381-ab44-bf7a9dc01a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:07.916942 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b21e5-a287-4381-ab44-bf7a9dc01a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.917223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b21e5-a287-4381-ab44-bf7a9dc01a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.917529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b21e5-a287-4381-ab44-bf7a9dc01a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.918231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b21e5-a287-4381-ab44-bf7a9dc01a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:07.919869 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b21e5-a287-4381-ab44-bf7a9dc01a5f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:07.933407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1086/2161] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:07.933407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac5b03-7247-4544-9d5d-b91f8d9c0dc3 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.933407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac5b03-7247-4544-9d5d-b91f8d9c0dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.933407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac5b03-7247-4544-9d5d-b91f8d9c0dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.952824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7dac5b03-7247-4544-9d5d-b91f8d9c0dc3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:07.954148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7dac5b03-7247-4544-9d5d-b91f8d9c0dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.963036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1087/2162] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:07.963835 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac5b03-7247-4544-9d5d-b91f8d9c0dc3 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.974533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac5b03-7247-4544-9d5d-b91f8d9c0dc3 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:07.981358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] The token cache is being invalidate because project 24305a12daa54c6c94ec0fdd234ab400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:07.981657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.982376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:07.982933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ac53f-69aa-4701-b389-c9a29b4ef19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:07.983436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ac53f-69aa-4701-b389-c9a29b4ef19f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:07.983803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ac53f-69aa-4701-b389-c9a29b4ef19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:07.984258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ac53f-69aa-4701-b389-c9a29b4ef19f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:07.985090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ac53f-69aa-4701-b389-c9a29b4ef19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:07.986963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1076/2163] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:07 2022] DELETE /identity/v3/projects/24305a12daa54c6c94ec0fdd234ab400 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:07.987265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ac53f-69aa-4701-b389-c9a29b4ef19f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:07.987825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:07.989529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:07.998962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1088/2164] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:07.999833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ac53f-69aa-4701-b389-c9a29b4ef19f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:08.001831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-549ac53f-69aa-4701-b389-c9a29b4ef19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.006974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c659c82-bd40-4ac8-9fff-255aab51f8b7 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:08.012555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:08.013177 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.013458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.013737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.015123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:08.015123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:08.019156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:08.019287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ac53f-69aa-4701-b389-c9a29b4ef19f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:08.023830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:08.024560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.025221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.025221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] PATH_INFO: `/v3/users/f0967244091c47f1991f90d4548aedd9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.026430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0967244091c47f1991f90d4548aedd9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:08.028992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:08.034340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1077/2165] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:08.035425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:08.037892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.053108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad89ef9-5d62-4cf9-aab3-ee304d7ead6b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:08.058461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157892eb-961a-4659-aae7-340e57b2d5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:08.059146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157892eb-961a-4659-aae7-340e57b2d5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.059415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157892eb-961a-4659-aae7-340e57b2d5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.059690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157892eb-961a-4659-aae7-340e57b2d5e3 admin admin] PATH_INFO: `/v3/projects/62212e0b0d374a8db1e0723bd470e061/users/4ed6bcd86f4f4b1bae640d3c996e84ce/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.077477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157892eb-961a-4659-aae7-340e57b2d5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62212e0b0d374a8db1e0723bd470e061, user_id=4ed6bcd86f4f4b1bae640d3c996e84ce, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:08.080401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157892eb-961a-4659-aae7-340e57b2d5e3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:08.154523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:08.159319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1078/2166] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:08 2022] PUT /identity/v3/projects/62212e0b0d374a8db1e0723bd470e061/users/4ed6bcd86f4f4b1bae640d3c996e84ce/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:08.169382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157892eb-961a-4659-aae7-340e57b2d5e3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:08.169382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-157892eb-961a-4659-aae7-340e57b2d5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.235926 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157892eb-961a-4659-aae7-340e57b2d5e3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:08.240761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:08.241395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.241695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.241980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] PATH_INFO: `/v3/projects/e01c531806484c6aac10a6e318e060c2` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.243144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01c531806484c6aac10a6e318e060c2)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:08.245096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:08.247583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0967244091c47f1991f90d4548aedd9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.248358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0967244091c47f1991f90d4548aedd9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.248973 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0967244091c47f1991f90d4548aedd9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.256077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0967244091c47f1991f90d4548aedd9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.270126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0967244091c47f1991f90d4548aedd9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.380807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1089/2167] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:07 2022] DELETE /identity/v3/users/f0967244091c47f1991f90d4548aedd9 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:08.381718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:08.383421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.399334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01c531806484c6aac10a6e318e060c2', 'request_id': 'req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.407318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01c531806484c6aac10a6e318e060c2', 'request_id': 'req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.416117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01c531806484c6aac10a6e318e060c2', 'request_id': 'req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.463388 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54cdeb-2bd3-4ef4-9742-a0347d6c085b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:08.472624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95c3e-0712-4828-a1de-ae73bb729d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:08.473523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95c3e-0712-4828-a1de-ae73bb729d1c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.473810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95c3e-0712-4828-a1de-ae73bb729d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.476303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95c3e-0712-4828-a1de-ae73bb729d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.477045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a95c3e-0712-4828-a1de-ae73bb729d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:08.479171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a95c3e-0712-4828-a1de-ae73bb729d1c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:08.496560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] The token cache is being invalidate because project e01c531806484c6aac10a6e318e060c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:08.497330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.498901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d818df98-eb88-45ce-8ecd-c8a2c7f4f03e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:08.501496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1079/2168] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:08 2022] DELETE /identity/v3/projects/e01c531806484c6aac10a6e318e060c2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:08.503109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda04c4-a964-486d-96b6-1adbfdf9fb51 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.503654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda04c4-a964-486d-96b6-1adbfdf9fb51 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.504048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda04c4-a964-486d-96b6-1adbfdf9fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.524308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:08.538628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5eda04c4-a964-486d-96b6-1adbfdf9fb51 None None] MFA Rules not processed for user `4ed6bcd86f4f4b1bae640d3c996e84ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:08.539985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5eda04c4-a964-486d-96b6-1adbfdf9fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.589097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1090/2169] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:08 2022] POST /identity/v3/projects => generated 363 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:08.591476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6f2bf-1661-434f-990e-0fa493735bae None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.591944 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6f2bf-1661-434f-990e-0fa493735bae None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.593265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6f2bf-1661-434f-990e-0fa493735bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.619766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1080/2170] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:08.621048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda04c4-a964-486d-96b6-1adbfdf9fb51 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:08.622709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5eda04c4-a964-486d-96b6-1adbfdf9fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.630018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-abd6f2bf-1661-434f-990e-0fa493735bae None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:08.634289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-abd6f2bf-1661-434f-990e-0fa493735bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.641037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda04c4-a964-486d-96b6-1adbfdf9fb51 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:08.724459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a4fe8-ce0f-4f9b-a44b-b9533810896a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:08.725195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a4fe8-ce0f-4f9b-a44b-b9533810896a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.725488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a4fe8-ce0f-4f9b-a44b-b9533810896a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.725770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a4fe8-ce0f-4f9b-a44b-b9533810896a admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.726459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a4fe8-ce0f-4f9b-a44b-b9533810896a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:08.728484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a4fe8-ce0f-4f9b-a44b-b9533810896a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:08.760984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1091/2171] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:08.760984 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6f2bf-1661-434f-990e-0fa493735bae None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:08.764295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-abd6f2bf-1661-434f-990e-0fa493735bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.780338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6f2bf-1661-434f-990e-0fa493735bae None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:08.787074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1799623-7a0b-40cb-8a15-4ad10c2eeef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:08.788122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1799623-7a0b-40cb-8a15-4ad10c2eeef4 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.788536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1799623-7a0b-40cb-8a15-4ad10c2eeef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.788807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1799623-7a0b-40cb-8a15-4ad10c2eeef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.789568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1799623-7a0b-40cb-8a15-4ad10c2eeef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:08.791802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1799623-7a0b-40cb-8a15-4ad10c2eeef4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:08.918866 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1081/2172] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:08 2022] POST /identity/v3/users => generated 282 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:08.920143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba8a79-5985-4980-91f8-058eb3d05eb7 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.920430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba8a79-5985-4980-91f8-058eb3d05eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.920701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba8a79-5985-4980-91f8-058eb3d05eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.939937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-53ba8a79-5985-4980-91f8-058eb3d05eb7 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:08.941149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-53ba8a79-5985-4980-91f8-058eb3d05eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.950067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1082/2173] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:08.951375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f781c-7200-4e3c-ae84-ace3bf0154d5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:08.951659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f781c-7200-4e3c-ae84-ace3bf0154d5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:08.951926 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f781c-7200-4e3c-ae84-ace3bf0154d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:08.972411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-656f781c-7200-4e3c-ae84-ace3bf0154d5 None None] MFA Rules not processed for user `8a512d59d7b4477b88f7bb9532141d06`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:08.973679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-656f781c-7200-4e3c-ae84-ace3bf0154d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:08.990545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1092/2174] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:08 2022] POST /identity/v3/projects => generated 387 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:08.991068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1799623-7a0b-40cb-8a15-4ad10c2eeef4 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.053150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1083/2175] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:09.054135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f781c-7200-4e3c-ae84-ace3bf0154d5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.061192 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1799623-7a0b-40cb-8a15-4ad10c2eeef4 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.068319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e213cb-fe40-45b6-9e14-df7c6bd7a446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.070437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e213cb-fe40-45b6-9e14-df7c6bd7a446 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.072348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e213cb-fe40-45b6-9e14-df7c6bd7a446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.072348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e213cb-fe40-45b6-9e14-df7c6bd7a446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.072348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e213cb-fe40-45b6-9e14-df7c6bd7a446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.073341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e213cb-fe40-45b6-9e14-df7c6bd7a446 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.082216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f781c-7200-4e3c-ae84-ace3bf0154d5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.085078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1093/2176] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:09.086029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e213cb-fe40-45b6-9e14-df7c6bd7a446 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.087292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d08cc-b4ba-4432-893e-2c769fbca15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.087658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c9e213cb-fe40-45b6-9e14-df7c6bd7a446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:09.088103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d08cc-b4ba-4432-893e-2c769fbca15c admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.088276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d08cc-b4ba-4432-893e-2c769fbca15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.088555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d08cc-b4ba-4432-893e-2c769fbca15c admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.089235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d08cc-b4ba-4432-893e-2c769fbca15c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:09.089542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d08cc-b4ba-4432-893e-2c769fbca15c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.091218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d08cc-b4ba-4432-893e-2c769fbca15c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.102434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e213cb-fe40-45b6-9e14-df7c6bd7a446 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.106019 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1084/2177] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:09.106371 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d08cc-b4ba-4432-893e-2c769fbca15c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.108170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.108878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.109098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.109375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] PATH_INFO: `/v3/projects/67b5ba4ab2034d58905342d90f663692` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.110727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b5ba4ab2034d58905342d90f663692)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.112684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.121650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d08cc-b4ba-4432-893e-2c769fbca15c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.143245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bcfa5-46e6-45a3-8589-c27abf416e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.143245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bcfa5-46e6-45a3-8589-c27abf416e61 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.143245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bcfa5-46e6-45a3-8589-c27abf416e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.143245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bcfa5-46e6-45a3-8589-c27abf416e61 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.150569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bcfa5-46e6-45a3-8589-c27abf416e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.156375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bcfa5-46e6-45a3-8589-c27abf416e61 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.246939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b5ba4ab2034d58905342d90f663692', 'request_id': 'req-430ad907-11e1-4d2e-98b2-ca06a7149baf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:09.254542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b5ba4ab2034d58905342d90f663692', 'request_id': 'req-430ad907-11e1-4d2e-98b2-ca06a7149baf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:09.275007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1085/2178] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:09 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:09.276017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bcfa5-46e6-45a3-8589-c27abf416e61 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.293740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bcfa5-46e6-45a3-8589-c27abf416e61 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.384281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43dee-c6e6-4b5c-9c7d-a61fbf6a904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.385434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43dee-c6e6-4b5c-9c7d-a61fbf6a904e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.385721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43dee-c6e6-4b5c-9c7d-a61fbf6a904e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.386038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43dee-c6e6-4b5c-9c7d-a61fbf6a904e admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.386739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b43dee-c6e6-4b5c-9c7d-a61fbf6a904e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.388706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b43dee-c6e6-4b5c-9c7d-a61fbf6a904e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.407983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1086/2179] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:09.409245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43dee-c6e6-4b5c-9c7d-a61fbf6a904e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.415150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b5ba4ab2034d58905342d90f663692', 'request_id': 'req-430ad907-11e1-4d2e-98b2-ca06a7149baf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:09.425044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43dee-c6e6-4b5c-9c7d-a61fbf6a904e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.430667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] The token cache is being invalidate because project 67b5ba4ab2034d58905342d90f663692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:09.431129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:09.431825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:09.434543 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1094/2180] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:09 2022] DELETE /identity/v3/projects/67b5ba4ab2034d58905342d90f663692 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:09.435370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.436755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:09.440632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50053699-9acd-4afe-b7ce-b538f08dd5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.441758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50053699-9acd-4afe-b7ce-b538f08dd5de admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.442498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50053699-9acd-4afe-b7ce-b538f08dd5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.442609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50053699-9acd-4afe-b7ce-b538f08dd5de admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.443203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50053699-9acd-4afe-b7ce-b538f08dd5de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.445266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50053699-9acd-4afe-b7ce-b538f08dd5de admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.468912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ad907-11e1-4d2e-98b2-ca06a7149baf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.476152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff1e80-2693-4f23-94f9-5109d6b8aa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.477062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff1e80-2693-4f23-94f9-5109d6b8aa90 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.477062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff1e80-2693-4f23-94f9-5109d6b8aa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.477347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff1e80-2693-4f23-94f9-5109d6b8aa90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.478060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff1e80-2693-4f23-94f9-5109d6b8aa90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.479709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff1e80-2693-4f23-94f9-5109d6b8aa90 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.496221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1095/2181] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:09.496221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff1e80-2693-4f23-94f9-5109d6b8aa90 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.496221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b7ff1e80-2693-4f23-94f9-5109d6b8aa90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:09.509254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff1e80-2693-4f23-94f9-5109d6b8aa90 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.514556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767801ce-9f6f-4647-a81d-45cc362f96c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.515235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767801ce-9f6f-4647-a81d-45cc362f96c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.515516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767801ce-9f6f-4647-a81d-45cc362f96c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.515818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767801ce-9f6f-4647-a81d-45cc362f96c3 admin admin] PATH_INFO: `/v3/projects/594f9b805c3748f0850f0c7be6524b44/users/3910367683114ce6a7e065fc039efc04/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.533698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767801ce-9f6f-4647-a81d-45cc362f96c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594f9b805c3748f0850f0c7be6524b44, user_id=3910367683114ce6a7e065fc039efc04, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.536789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767801ce-9f6f-4647-a81d-45cc362f96c3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.582995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1087/2182] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:09 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:09.584827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4981d7-b4ca-488b-b453-a96788f792b1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.585120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4981d7-b4ca-488b-b453-a96788f792b1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.585391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4981d7-b4ca-488b-b453-a96788f792b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.628160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-be4981d7-b4ca-488b-b453-a96788f792b1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:09.630501 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-be4981d7-b4ca-488b-b453-a96788f792b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:09.640598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1096/2183] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:09 2022] PUT /identity/v3/projects/594f9b805c3748f0850f0c7be6524b44/users/3910367683114ce6a7e065fc039efc04/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:09.641655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767801ce-9f6f-4647-a81d-45cc362f96c3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.709248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1088/2184] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:09.710599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4981d7-b4ca-488b-b453-a96788f792b1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.712568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-be4981d7-b4ca-488b-b453-a96788f792b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:09.726585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4981d7-b4ca-488b-b453-a96788f792b1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.732029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad348e8-5fbe-44ab-9cf5-e1dd80564836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.732651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad348e8-5fbe-44ab-9cf5-e1dd80564836 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.732919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad348e8-5fbe-44ab-9cf5-e1dd80564836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.733190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad348e8-5fbe-44ab-9cf5-e1dd80564836 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.733830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad348e8-5fbe-44ab-9cf5-e1dd80564836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.735793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad348e8-5fbe-44ab-9cf5-e1dd80564836 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.737669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767801ce-9f6f-4647-a81d-45cc362f96c3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.742445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579c51a-479f-4e9c-8fdb-e1337c2aa197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.743109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579c51a-479f-4e9c-8fdb-e1337c2aa197 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.743389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579c51a-479f-4e9c-8fdb-e1337c2aa197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.743683 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579c51a-479f-4e9c-8fdb-e1337c2aa197 admin admin] PATH_INFO: `/v3/projects/78698aa91264475da1d3877f6af8e620/users/ce1098107ad24369811f138fa3a3c399/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.761784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579c51a-479f-4e9c-8fdb-e1337c2aa197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78698aa91264475da1d3877f6af8e620, user_id=ce1098107ad24369811f138fa3a3c399, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.765753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579c51a-479f-4e9c-8fdb-e1337c2aa197 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:09.835511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1089/2185] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:09 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:09.835511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012ee0e-ca1c-47b2-b0e8-f2cec3bda889 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.835511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012ee0e-ca1c-47b2-b0e8-f2cec3bda889 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.835511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012ee0e-ca1c-47b2-b0e8-f2cec3bda889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.865633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b012ee0e-ca1c-47b2-b0e8-f2cec3bda889 None None] MFA Rules not processed for user `3910367683114ce6a7e065fc039efc04`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:09.867731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b012ee0e-ca1c-47b2-b0e8-f2cec3bda889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:09.872080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1097/2186] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:09 2022] PUT /identity/v3/projects/78698aa91264475da1d3877f6af8e620/users/ce1098107ad24369811f138fa3a3c399/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:09.876690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc6f1-e4a0-4e8e-8c9a-311b7a598cd8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.876690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc6f1-e4a0-4e8e-8c9a-311b7a598cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.876690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc6f1-e4a0-4e8e-8c9a-311b7a598cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.896493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b03cc6f1-e4a0-4e8e-8c9a-311b7a598cd8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:09.897975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b03cc6f1-e4a0-4e8e-8c9a-311b7a598cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:09.951968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1090/2187] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:09.953330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012ee0e-ca1c-47b2-b0e8-f2cec3bda889 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.970608 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1098/2188] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:09.972064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc6f1-e4a0-4e8e-8c9a-311b7a598cd8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:09.982558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc6f1-e4a0-4e8e-8c9a-311b7a598cd8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:09.987268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b4115-1da8-4742-8cde-76670c6f2f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:09.987865 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b4115-1da8-4742-8cde-76670c6f2f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:09.988159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b4115-1da8-4742-8cde-76670c6f2f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:09.988415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b4115-1da8-4742-8cde-76670c6f2f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:09.989067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b4115-1da8-4742-8cde-76670c6f2f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:09.990739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b4115-1da8-4742-8cde-76670c6f2f96 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.002641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1099/2189] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:10.003607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b4115-1da8-4742-8cde-76670c6f2f96 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.007413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fe8b4115-1da8-4742-8cde-76670c6f2f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:10.026625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012ee0e-ca1c-47b2-b0e8-f2cec3bda889 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.032155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048a8d1-dbda-4456-a153-5dd78c540720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.032769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048a8d1-dbda-4456-a153-5dd78c540720 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.033051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048a8d1-dbda-4456-a153-5dd78c540720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.033328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048a8d1-dbda-4456-a153-5dd78c540720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.034040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048a8d1-dbda-4456-a153-5dd78c540720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.035928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048a8d1-dbda-4456-a153-5dd78c540720 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.047189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1091/2190] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:10.048044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048a8d1-dbda-4456-a153-5dd78c540720 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.058285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048a8d1-dbda-4456-a153-5dd78c540720 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.063633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f52234-bff3-44ef-aa64-26871f7d5e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.064240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f52234-bff3-44ef-aa64-26871f7d5e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.064520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f52234-bff3-44ef-aa64-26871f7d5e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.064817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f52234-bff3-44ef-aa64-26871f7d5e0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.065366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b4115-1da8-4742-8cde-76670c6f2f96 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.065478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f52234-bff3-44ef-aa64-26871f7d5e0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:10.065736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f52234-bff3-44ef-aa64-26871f7d5e0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.067443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f52234-bff3-44ef-aa64-26871f7d5e0b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.069486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c95f3-2ec3-4d09-96c4-a2d9ebc11127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.070147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c95f3-2ec3-4d09-96c4-a2d9ebc11127 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.070666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c95f3-2ec3-4d09-96c4-a2d9ebc11127 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.070950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c95f3-2ec3-4d09-96c4-a2d9ebc11127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.080586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c95f3-2ec3-4d09-96c4-a2d9ebc11127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.080975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1092/2191] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:10.082002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f52234-bff3-44ef-aa64-26871f7d5e0b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.082358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c95f3-2ec3-4d09-96c4-a2d9ebc11127 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.092204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f52234-bff3-44ef-aa64-26871f7d5e0b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.092724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1100/2192] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:10.093534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c95f3-2ec3-4d09-96c4-a2d9ebc11127 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.097281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b8f64-e67c-45a4-81a0-e08a7d7f5602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.098063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b8f64-e67c-45a4-81a0-e08a7d7f5602 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.098239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b8f64-e67c-45a4-81a0-e08a7d7f5602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.098516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b8f64-e67c-45a4-81a0-e08a7d7f5602 admin admin] PATH_INFO: `/v3/projects/78698aa91264475da1d3877f6af8e620/users/ce1098107ad24369811f138fa3a3c399/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.101056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b8f64-e67c-45a4-81a0-e08a7d7f5602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78698aa91264475da1d3877f6af8e620, user_id=ce1098107ad24369811f138fa3a3c399, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.103760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b8f64-e67c-45a4-81a0-e08a7d7f5602 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.105025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c95f3-2ec3-4d09-96c4-a2d9ebc11127 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.109535 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b207b8-5651-455b-bcbc-77543f1a58c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.110190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b207b8-5651-455b-bcbc-77543f1a58c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.110456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b207b8-5651-455b-bcbc-77543f1a58c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.110733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b207b8-5651-455b-bcbc-77543f1a58c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.111390 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b207b8-5651-455b-bcbc-77543f1a58c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.113162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b207b8-5651-455b-bcbc-77543f1a58c5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.115622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1093/2193] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:10 2022] PUT /identity/v3/projects/78698aa91264475da1d3877f6af8e620/users/ce1098107ad24369811f138fa3a3c399/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:10.116440 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b8f64-e67c-45a4-81a0-e08a7d7f5602 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.137656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1101/2194] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:10.137656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ceb0c-6b4b-4a21-b0fe-09e22c184bc8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.137656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ceb0c-6b4b-4a21-b0fe-09e22c184bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.137656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ceb0c-6b4b-4a21-b0fe-09e22c184bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.166784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-065ceb0c-6b4b-4a21-b0fe-09e22c184bc8 None None] MFA Rules not processed for user `ce1098107ad24369811f138fa3a3c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:10.168757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-065ceb0c-6b4b-4a21-b0fe-09e22c184bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:10.196077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b8f64-e67c-45a4-81a0-e08a7d7f5602 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.202914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131638f-62ad-4d91-9269-c75879df469c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.204688 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131638f-62ad-4d91-9269-c75879df469c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.204688 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131638f-62ad-4d91-9269-c75879df469c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.204688 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131638f-62ad-4d91-9269-c75879df469c admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.204949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131638f-62ad-4d91-9269-c75879df469c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.207726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131638f-62ad-4d91-9269-c75879df469c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.265513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1102/2195] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:10.266545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ceb0c-6b4b-4a21-b0fe-09e22c184bc8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.280889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ceb0c-6b4b-4a21-b0fe-09e22c184bc8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.287121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ca803-f6ca-40ec-8998-d21675a96ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.288063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ca803-f6ca-40ec-8998-d21675a96ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.288647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ca803-f6ca-40ec-8998-d21675a96ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.288983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ca803-f6ca-40ec-8998-d21675a96ea5 admin admin] PATH_INFO: `/v3/projects/92e4fbb4e1484356860a59f8782adaac/users/da3ec93bddc0425da2974f821c258ef5/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.307111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ca803-f6ca-40ec-8998-d21675a96ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e4fbb4e1484356860a59f8782adaac, user_id=da3ec93bddc0425da2974f821c258ef5, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.310056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1094/2196] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:10 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:10.318148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131638f-62ad-4d91-9269-c75879df469c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.318288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ca803-f6ca-40ec-8998-d21675a96ea5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.318367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8131638f-62ad-4d91-9269-c75879df469c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:10.381966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131638f-62ad-4d91-9269-c75879df469c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.387576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876fad5-b685-4295-8d5d-a4cc644f209f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.388213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876fad5-b685-4295-8d5d-a4cc644f209f service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.388490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876fad5-b685-4295-8d5d-a4cc644f209f service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.388759 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876fad5-b685-4295-8d5d-a4cc644f209f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.422507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1103/2197] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:10 2022] PUT /identity/v3/projects/92e4fbb4e1484356860a59f8782adaac/users/da3ec93bddc0425da2974f821c258ef5/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:10.423630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ca803-f6ca-40ec-8998-d21675a96ea5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.465546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876fad5-b685-4295-8d5d-a4cc644f209f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.467025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876fad5-b685-4295-8d5d-a4cc644f209f service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.477981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1095/2198] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:10.490537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd41c58-e2f6-4519-b0d2-d586187dd1ad None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.490743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd41c58-e2f6-4519-b0d2-d586187dd1ad None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.491454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd41c58-e2f6-4519-b0d2-d586187dd1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.525806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ca803-f6ca-40ec-8998-d21675a96ea5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.532936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79512b62-20dd-49d3-abb1-7d6e0e425ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.533915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79512b62-20dd-49d3-abb1-7d6e0e425ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.534092 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79512b62-20dd-49d3-abb1-7d6e0e425ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.534465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79512b62-20dd-49d3-abb1-7d6e0e425ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.535109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79512b62-20dd-49d3-abb1-7d6e0e425ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.537653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79512b62-20dd-49d3-abb1-7d6e0e425ad4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.539522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7bd41c58-e2f6-4519-b0d2-d586187dd1ad None None] MFA Rules not processed for user `da3ec93bddc0425da2974f821c258ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:10.540758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7bd41c58-e2f6-4519-b0d2-d586187dd1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:10.628460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1096/2199] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:10.648815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd41c58-e2f6-4519-b0d2-d586187dd1ad None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.659872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd41c58-e2f6-4519-b0d2-d586187dd1ad None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.664766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.665391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.665681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.665978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] PATH_INFO: `/v3/users/4ed6bcd86f4f4b1bae640d3c996e84ce` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.667151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed6bcd86f4f4b1bae640d3c996e84ce)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.669101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.678683 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:10.692906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1104/2200] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:10 2022] POST /identity/v3/users => generated 284 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:10.692906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79512b62-20dd-49d3-abb1-7d6e0e425ad4 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.749380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79512b62-20dd-49d3-abb1-7d6e0e425ad4 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.753064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf565dc9-5f7d-45e5-a38a-4cd329532e28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.753666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf565dc9-5f7d-45e5-a38a-4cd329532e28 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.753977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf565dc9-5f7d-45e5-a38a-4cd329532e28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.754267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf565dc9-5f7d-45e5-a38a-4cd329532e28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.763249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf565dc9-5f7d-45e5-a38a-4cd329532e28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.764794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf565dc9-5f7d-45e5-a38a-4cd329532e28 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.775417 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1105/2201] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:10.776217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf565dc9-5f7d-45e5-a38a-4cd329532e28 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.788266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf565dc9-5f7d-45e5-a38a-4cd329532e28 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.793589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fa203-20f5-4751-92e2-0c07abfe3449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.794047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fa203-20f5-4751-92e2-0c07abfe3449 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.794607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fa203-20f5-4751-92e2-0c07abfe3449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.794887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fa203-20f5-4751-92e2-0c07abfe3449 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.795541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fa203-20f5-4751-92e2-0c07abfe3449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.797284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fa203-20f5-4751-92e2-0c07abfe3449 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.811115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1106/2202] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:10.811115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fa203-20f5-4751-92e2-0c07abfe3449 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:10.819289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fa203-20f5-4751-92e2-0c07abfe3449 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:10.824668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:10.825288 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:10.825572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:10.825849 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] PATH_INFO: `/v3/users/da3ec93bddc0425da2974f821c258ef5` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:10.827022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3ec93bddc0425da2974f821c258ef5)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:10.828958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:10.985101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed6bcd86f4f4b1bae640d3c996e84ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:10.985760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed6bcd86f4f4b1bae640d3c996e84ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:10.986503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed6bcd86f4f4b1bae640d3c996e84ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:11.117655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed6bcd86f4f4b1bae640d3c996e84ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:11.126236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed6bcd86f4f4b1bae640d3c996e84ce'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:11.132982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3ec93bddc0425da2974f821c258ef5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:11.133631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3ec93bddc0425da2974f821c258ef5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:11.134251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3ec93bddc0425da2974f821c258ef5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:11.141177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3ec93bddc0425da2974f821c258ef5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:11.145075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1097/2203] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:10 2022] DELETE /identity/v3/users/4ed6bcd86f4f4b1bae640d3c996e84ce => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:11.145835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:11.150043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:11.155852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3ec93bddc0425da2974f821c258ef5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:11.212006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05eb63-884f-45e2-978a-5e07f9e6804e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:11.217550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cb8db-441a-4166-a3ea-a5cdabadf345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:11.218187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cb8db-441a-4166-a3ea-a5cdabadf345 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:11.218460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cb8db-441a-4166-a3ea-a5cdabadf345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:11.218730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cb8db-441a-4166-a3ea-a5cdabadf345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:11.219518 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cb8db-441a-4166-a3ea-a5cdabadf345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:11.221116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cb8db-441a-4166-a3ea-a5cdabadf345 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:11.232105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1098/2204] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:11.233369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07fdc9-7cb0-47bb-bef6-fc30ed846d15 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:11.233655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07fdc9-7cb0-47bb-bef6-fc30ed846d15 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:11.233956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07fdc9-7cb0-47bb-bef6-fc30ed846d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:11.252060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-db07fdc9-7cb0-47bb-bef6-fc30ed846d15 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:11.253268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-db07fdc9-7cb0-47bb-bef6-fc30ed846d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:11.261045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1099/2205] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:11.262015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07fdc9-7cb0-47bb-bef6-fc30ed846d15 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:11.263126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-db07fdc9-7cb0-47bb-bef6-fc30ed846d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:11.274951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07fdc9-7cb0-47bb-bef6-fc30ed846d15 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:11.278639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1107/2206] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:10 2022] DELETE /identity/v3/users/da3ec93bddc0425da2974f821c258ef5 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:11.281641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:11.281737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8576f0-fda2-402c-abe8-b34d46fb84cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:11.281737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8576f0-fda2-402c-abe8-b34d46fb84cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:11.281737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8576f0-fda2-402c-abe8-b34d46fb84cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:11.281737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8576f0-fda2-402c-abe8-b34d46fb84cd admin admin] PATH_INFO: `/v3/projects/151ecdfda6ca4ee6824d9f4a7d646520/users/b43e457ebcfd4ff29b13f3df2fc4e1e7/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:11.286442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:11.298370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8576f0-fda2-402c-abe8-b34d46fb84cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=151ecdfda6ca4ee6824d9f4a7d646520, user_id=b43e457ebcfd4ff29b13f3df2fc4e1e7, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:11.301076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8576f0-fda2-402c-abe8-b34d46fb84cd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:11.344817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6308da-c8f2-491b-8906-2b20c114d803 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:11.348432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605dcfb-a0fc-4a27-af3d-f536c0695e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:11.349027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605dcfb-a0fc-4a27-af3d-f536c0695e46 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:11.349303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605dcfb-a0fc-4a27-af3d-f536c0695e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:11.349572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605dcfb-a0fc-4a27-af3d-f536c0695e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:11.405801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1100/2207] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:11 2022] PUT /identity/v3/projects/151ecdfda6ca4ee6824d9f4a7d646520/users/b43e457ebcfd4ff29b13f3df2fc4e1e7/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:11.407355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce77f4-9192-47cb-bd53-0930edf8beda None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:11.407797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce77f4-9192-47cb-bd53-0930edf8beda None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:11.408287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce77f4-9192-47cb-bd53-0930edf8beda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:11.422616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605dcfb-a0fc-4a27-af3d-f536c0695e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:11.424065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605dcfb-a0fc-4a27-af3d-f536c0695e46 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:11.427442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:11.430707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-30ce77f4-9192-47cb-bd53-0930edf8beda None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:11.431980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-30ce77f4-9192-47cb-bd53-0930edf8beda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:11.437754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1108/2208] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:11.439226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fe3a5-0a2d-4746-a0f2-b08c9e9f6f3c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:11.439734 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fe3a5-0a2d-4746-a0f2-b08c9e9f6f3c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:11.440157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fe3a5-0a2d-4746-a0f2-b08c9e9f6f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:11.469550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ad3fe3a5-0a2d-4746-a0f2-b08c9e9f6f3c None None] MFA Rules not processed for user `b43e457ebcfd4ff29b13f3df2fc4e1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:11.470431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ad3fe3a5-0a2d-4746-a0f2-b08c9e9f6f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:11.499297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1101/2209] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:11.509127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce77f4-9192-47cb-bd53-0930edf8beda None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:11.510687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-30ce77f4-9192-47cb-bd53-0930edf8beda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:11.551009 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1109/2210] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:11.565281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fe3a5-0a2d-4746-a0f2-b08c9e9f6f3c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:11.568677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ad3fe3a5-0a2d-4746-a0f2-b08c9e9f6f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:11.592843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce77f4-9192-47cb-bd53-0930edf8beda None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:11.601285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cccd7-f5a9-43ae-a448-845c13a77991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:11.602062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cccd7-f5a9-43ae-a448-845c13a77991 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:11.602856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cccd7-f5a9-43ae-a448-845c13a77991 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:11.603223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cccd7-f5a9-43ae-a448-845c13a77991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:11.620098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cccd7-f5a9-43ae-a448-845c13a77991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:11.622543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cccd7-f5a9-43ae-a448-845c13a77991 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:11.642488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1102/2211] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:27:11.682283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fe3a5-0a2d-4746-a0f2-b08c9e9f6f3c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:11.688972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68f924-bc78-4e33-852e-8e4f504b79b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f214c2b77d4cbab2f15122ba21bf38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:11.689623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68f924-bc78-4e33-852e-8e4f504b79b0 service glance] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:11.689941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68f924-bc78-4e33-852e-8e4f504b79b0 service glance] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:11.690226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68f924-bc78-4e33-852e-8e4f504b79b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:11.705495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68f924-bc78-4e33-852e-8e4f504b79b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:11.707465 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68f924-bc78-4e33-852e-8e4f504b79b0 service glance] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:11.730239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1110/2212] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:27:12.094650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cccd7-f5a9-43ae-a448-845c13a77991 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:12.096237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-760cccd7-f5a9-43ae-a448-845c13a77991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:12.122499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cccd7-f5a9-43ae-a448-845c13a77991 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:12.154761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:12.155512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:12.155863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:12.156208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] PATH_INFO: `/v3/projects/62212e0b0d374a8db1e0723bd470e061` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:12.158337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] RBAC: Authorizing `identity:delete_project(project_id=62212e0b0d374a8db1e0723bd470e061)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:12.160612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:12.324453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62212e0b0d374a8db1e0723bd470e061', 'request_id': 'req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.330125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62212e0b0d374a8db1e0723bd470e061', 'request_id': 'req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.342653 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62212e0b0d374a8db1e0723bd470e061', 'request_id': 'req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.350454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68f924-bc78-4e33-852e-8e4f504b79b0 service glance] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:12.353060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cb68f924-bc78-4e33-852e-8e4f504b79b0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:12.373637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68f924-bc78-4e33-852e-8e4f504b79b0 service glance] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:12.435085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:12.435723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:12.436010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:12.436286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] PATH_INFO: `/v3/projects/92e4fbb4e1484356860a59f8782adaac` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:12.437516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e4fbb4e1484356860a59f8782adaac)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:12.439492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:12.449394 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] The token cache is being invalidate because project 62212e0b0d374a8db1e0723bd470e061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:12.449685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.450376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.454937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1103/2213] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:12 2022] DELETE /identity/v3/projects/62212e0b0d374a8db1e0723bd470e061 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:12.456363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:12.457448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:12.471169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d10a10-a362-4d97-b2f2-2fe525a3e09c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:12.476391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:12.477026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:12.477306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:12.477579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] PATH_INFO: `/v3/users/ce1098107ad24369811f138fa3a3c399` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:12.478767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce1098107ad24369811f138fa3a3c399)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:12.480746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:12.686116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e4fbb4e1484356860a59f8782adaac', 'request_id': 'req-ee920d32-2381-4f64-9dee-83ec5ed42f76'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.687614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce1098107ad24369811f138fa3a3c399'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.688470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce1098107ad24369811f138fa3a3c399'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.689251 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce1098107ad24369811f138fa3a3c399'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.692551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e4fbb4e1484356860a59f8782adaac', 'request_id': 'req-ee920d32-2381-4f64-9dee-83ec5ed42f76'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.820968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e4fbb4e1484356860a59f8782adaac', 'request_id': 'req-ee920d32-2381-4f64-9dee-83ec5ed42f76'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.841641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] The token cache is being invalidate because project 92e4fbb4e1484356860a59f8782adaac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:12.842262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.851998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ee920d32-2381-4f64-9dee-83ec5ed42f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.851998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1111/2214] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:12 2022] DELETE /identity/v3/projects/92e4fbb4e1484356860a59f8782adaac => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:12.851998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcdcfd-b2e6-40d1-9830-5bca7a605f03 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:12.851998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcdcfd-b2e6-40d1-9830-5bca7a605f03 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:12.851998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcdcfd-b2e6-40d1-9830-5bca7a605f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:12.881486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3bfcdcfd-b2e6-40d1-9830-5bca7a605f03 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:12.883752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3bfcdcfd-b2e6-40d1-9830-5bca7a605f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:12.947874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1112/2215] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:12.955978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4d3c4-3cf1-4e6f-8f07-440d15873fd5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:12.956354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4d3c4-3cf1-4e6f-8f07-440d15873fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:12.956636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4d3c4-3cf1-4e6f-8f07-440d15873fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:12.969437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce1098107ad24369811f138fa3a3c399'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.978083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce1098107ad24369811f138fa3a3c399'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:12.987153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bfc4d3c4-3cf1-4e6f-8f07-440d15873fd5 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:12.988026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bfc4d3c4-3cf1-4e6f-8f07-440d15873fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:13.001393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1104/2216] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:12 2022] DELETE /identity/v3/users/ce1098107ad24369811f138fa3a3c399 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:13.001913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1113/2217] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:13.006139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:13.009078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:13.107174 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1617b-8b72-4b9f-8a77-7e8b8775b447 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:13.113299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad4ec-1dcb-4a2c-8ef7-61da6145719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:13.114138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad4ec-1dcb-4a2c-8ef7-61da6145719f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:13.114490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad4ec-1dcb-4a2c-8ef7-61da6145719f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:13.115063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad4ec-1dcb-4a2c-8ef7-61da6145719f admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:13.115559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad4ec-1dcb-4a2c-8ef7-61da6145719f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:13.115928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad4ec-1dcb-4a2c-8ef7-61da6145719f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:13.117648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad4ec-1dcb-4a2c-8ef7-61da6145719f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:13.134595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1105/2218] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:13.138778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4d3c4-3cf1-4e6f-8f07-440d15873fd5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:13.150158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4d3c4-3cf1-4e6f-8f07-440d15873fd5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:13.156310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74710ec8-f9b6-49ca-b5f4-6d69c619f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:13.157478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74710ec8-f9b6-49ca-b5f4-6d69c619f16a admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:13.157478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74710ec8-f9b6-49ca-b5f4-6d69c619f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:13.157795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74710ec8-f9b6-49ca-b5f4-6d69c619f16a admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:13.158686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74710ec8-f9b6-49ca-b5f4-6d69c619f16a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:13.160992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74710ec8-f9b6-49ca-b5f4-6d69c619f16a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:13.343260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1114/2219] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:13 2022] POST /identity/v3/projects => generated 397 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:13.347646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad4ec-1dcb-4a2c-8ef7-61da6145719f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:13.359941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad4ec-1dcb-4a2c-8ef7-61da6145719f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:13.441722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e0021-8789-4cf4-85df-d6f8cbc7384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:13.443444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e0021-8789-4cf4-85df-d6f8cbc7384d admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:13.443886 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e0021-8789-4cf4-85df-d6f8cbc7384d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:13.444821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e0021-8789-4cf4-85df-d6f8cbc7384d admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:13.445721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e0021-8789-4cf4-85df-d6f8cbc7384d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:13.448593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e0021-8789-4cf4-85df-d6f8cbc7384d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:13.550552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1106/2220] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:13 2022] POST /identity/v3/users => generated 299 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:13.554568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74710ec8-f9b6-49ca-b5f4-6d69c619f16a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:13.566934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74710ec8-f9b6-49ca-b5f4-6d69c619f16a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:13.572698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6c92f-e8dc-422b-954f-16ed14040dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:13.578064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6c92f-e8dc-422b-954f-16ed14040dab admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:13.578064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6c92f-e8dc-422b-954f-16ed14040dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:13.578064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6c92f-e8dc-422b-954f-16ed14040dab admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:13.578064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6c92f-e8dc-422b-954f-16ed14040dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:13.578064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6c92f-e8dc-422b-954f-16ed14040dab admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:13.589258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1115/2221] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:13.594096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e0021-8789-4cf4-85df-d6f8cbc7384d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:13.607881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e0021-8789-4cf4-85df-d6f8cbc7384d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:13.613045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3ac8a-8133-4267-978a-21d601b5b984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:13.613667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3ac8a-8133-4267-978a-21d601b5b984 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:13.613989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3ac8a-8133-4267-978a-21d601b5b984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:13.614267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3ac8a-8133-4267-978a-21d601b5b984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:13.614957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3ac8a-8133-4267-978a-21d601b5b984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:13.617175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3ac8a-8133-4267-978a-21d601b5b984 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:13.630624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1107/2222] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:13.634899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6c92f-e8dc-422b-954f-16ed14040dab admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:13.645711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6c92f-e8dc-422b-954f-16ed14040dab admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:13.650708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43380a-3b54-4e56-9b9b-20e8e8ad3a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:13.651370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43380a-3b54-4e56-9b9b-20e8e8ad3a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:13.651648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43380a-3b54-4e56-9b9b-20e8e8ad3a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:13.651924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43380a-3b54-4e56-9b9b-20e8e8ad3a70 admin admin] PATH_INFO: `/v3/projects/d977d4d8926b4a97877152b8d2326c38/users/223cf8031506481c976b3c1a0c424fc7/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:13.670504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43380a-3b54-4e56-9b9b-20e8e8ad3a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d977d4d8926b4a97877152b8d2326c38, user_id=223cf8031506481c976b3c1a0c424fc7, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:13.673444 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43380a-3b54-4e56-9b9b-20e8e8ad3a70 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:13.777120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1116/2223] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:13 2022] PUT /identity/v3/projects/d977d4d8926b4a97877152b8d2326c38/users/223cf8031506481c976b3c1a0c424fc7/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:13.789319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b351391-6177-420f-8f68-c11f73a7d36a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:13.789813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b351391-6177-420f-8f68-c11f73a7d36a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:13.789924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b351391-6177-420f-8f68-c11f73a7d36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:13.833741 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0b351391-6177-420f-8f68-c11f73a7d36a None None] MFA Rules not processed for user `223cf8031506481c976b3c1a0c424fc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:13.835187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0b351391-6177-420f-8f68-c11f73a7d36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:13.879999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43380a-3b54-4e56-9b9b-20e8e8ad3a70 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:13.884621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3f43380a-3b54-4e56-9b9b-20e8e8ad3a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:13.933944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1108/2224] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:13.980159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:13.983131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b351391-6177-420f-8f68-c11f73a7d36a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:13.985125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43380a-3b54-4e56-9b9b-20e8e8ad3a70 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:13.986409 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0b351391-6177-420f-8f68-c11f73a7d36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:13.992602 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:13.993130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:13.993402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:13.993678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] PATH_INFO: `/v3/projects/78698aa91264475da1d3877f6af8e620` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:13.994933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78698aa91264475da1d3877f6af8e620)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:13.997031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:14.057070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b351391-6177-420f-8f68-c11f73a7d36a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:14.061090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2222bb6-f53b-4d6a-8c05-23c3f770283b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:14.061700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2222bb6-f53b-4d6a-8c05-23c3f770283b service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:14.062016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2222bb6-f53b-4d6a-8c05-23c3f770283b service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:14.062295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2222bb6-f53b-4d6a-8c05-23c3f770283b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:14.071477 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2222bb6-f53b-4d6a-8c05-23c3f770283b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:14.072975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2222bb6-f53b-4d6a-8c05-23c3f770283b service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:14.082702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1109/2225] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:14.285524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78698aa91264475da1d3877f6af8e620', 'request_id': 'req-cb79118c-461e-4083-8762-af180c334f97'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:14.301086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78698aa91264475da1d3877f6af8e620', 'request_id': 'req-cb79118c-461e-4083-8762-af180c334f97'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:14.468048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78698aa91264475da1d3877f6af8e620', 'request_id': 'req-cb79118c-461e-4083-8762-af180c334f97'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:14.483433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] The token cache is being invalidate because project 78698aa91264475da1d3877f6af8e620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:14.483727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:14.487251 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cb79118c-461e-4083-8762-af180c334f97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:14.493922 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1117/2226] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:13 2022] DELETE /identity/v3/projects/78698aa91264475da1d3877f6af8e620 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:14.499887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c898141-4b9a-4535-848a-e8f4968cee25 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:14.500480 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c898141-4b9a-4535-848a-e8f4968cee25 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:14.501126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c898141-4b9a-4535-848a-e8f4968cee25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:14.554755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9c898141-4b9a-4535-848a-e8f4968cee25 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:14.557004 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9c898141-4b9a-4535-848a-e8f4968cee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:14.587847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:14.706273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1110/2227] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:14.716601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401bfa9-2221-48a2-b67f-d8a84eae8f4f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:14.717523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401bfa9-2221-48a2-b67f-d8a84eae8f4f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:14.717764 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401bfa9-2221-48a2-b67f-d8a84eae8f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:14.753697 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f401bfa9-2221-48a2-b67f-d8a84eae8f4f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:14.753901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f401bfa9-2221-48a2-b67f-d8a84eae8f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:14.764253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1118/2228] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:14.768358 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c898141-4b9a-4535-848a-e8f4968cee25 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:14.781090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c898141-4b9a-4535-848a-e8f4968cee25 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:14.785021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1f415-8aa4-42b4-ac56-908d9fbed026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:14.786213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1f415-8aa4-42b4-ac56-908d9fbed026 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:14.786506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1f415-8aa4-42b4-ac56-908d9fbed026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:14.786784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1f415-8aa4-42b4-ac56-908d9fbed026 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:14.787455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1f415-8aa4-42b4-ac56-908d9fbed026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:14.787762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1f415-8aa4-42b4-ac56-908d9fbed026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:14.789599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1f415-8aa4-42b4-ac56-908d9fbed026 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:14.828106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1111/2229] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:14.832961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401bfa9-2221-48a2-b67f-d8a84eae8f4f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:14.843328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401bfa9-2221-48a2-b67f-d8a84eae8f4f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:14.848708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7e7a6-1445-453b-96e3-45e600f7c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:14.849345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7e7a6-1445-453b-96e3-45e600f7c1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:14.849626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7e7a6-1445-453b-96e3-45e600f7c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:14.849923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7e7a6-1445-453b-96e3-45e600f7c1b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:14.850572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7e7a6-1445-453b-96e3-45e600f7c1b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:14.852529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7e7a6-1445-453b-96e3-45e600f7c1b0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:15.003098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1119/2230] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:14 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:15.003822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1f415-8aa4-42b4-ac56-908d9fbed026 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:15.022184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1f415-8aa4-42b4-ac56-908d9fbed026 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:15.100538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0f988-abf0-48ff-920a-9efa7835d33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:15.101205 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0f988-abf0-48ff-920a-9efa7835d33c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:15.101489 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0f988-abf0-48ff-920a-9efa7835d33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:15.101765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0f988-abf0-48ff-920a-9efa7835d33c admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:15.102438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0f988-abf0-48ff-920a-9efa7835d33c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:15.104582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0f988-abf0-48ff-920a-9efa7835d33c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:15.235609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1112/2231] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:15 2022] POST /identity/v3/users => generated 284 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:15.236873 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7e7a6-1445-453b-96e3-45e600f7c1b0 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:15.247782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7e7a6-1445-453b-96e3-45e600f7c1b0 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:15.253049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9a079-f3e5-4d8c-b329-d978940b71a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:15.254287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9a079-f3e5-4d8c-b329-d978940b71a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:15.254502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9a079-f3e5-4d8c-b329-d978940b71a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:15.254769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9a079-f3e5-4d8c-b329-d978940b71a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:15.255428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9a079-f3e5-4d8c-b329-d978940b71a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:15.257035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9a079-f3e5-4d8c-b329-d978940b71a1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:15.272322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1120/2232] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:15.276384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0f988-abf0-48ff-920a-9efa7835d33c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:15.288320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0f988-abf0-48ff-920a-9efa7835d33c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:15.293761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a3d78-a473-415d-ae07-3e21cde9455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:15.298076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a3d78-a473-415d-ae07-3e21cde9455f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:15.298076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a3d78-a473-415d-ae07-3e21cde9455f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:15.298076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a3d78-a473-415d-ae07-3e21cde9455f admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:15.298076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a3d78-a473-415d-ae07-3e21cde9455f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:15.298076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a3d78-a473-415d-ae07-3e21cde9455f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:15.309145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1113/2233] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:27:15.313165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9a079-f3e5-4d8c-b329-d978940b71a1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:15.323746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9a079-f3e5-4d8c-b329-d978940b71a1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:15.329095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747dd0d-fa67-4d9f-8667-ca4eb8eb6344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:15.329924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747dd0d-fa67-4d9f-8667-ca4eb8eb6344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:15.330211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747dd0d-fa67-4d9f-8667-ca4eb8eb6344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:15.330493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747dd0d-fa67-4d9f-8667-ca4eb8eb6344 admin admin] PATH_INFO: `/v3/projects/0cea7bd224ee4210b5b2fb4fc742b7e6/users/97fbe5a69283452e9a3da3b25d3156a9/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:15.347495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747dd0d-fa67-4d9f-8667-ca4eb8eb6344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cea7bd224ee4210b5b2fb4fc742b7e6, user_id=97fbe5a69283452e9a3da3b25d3156a9, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:15.350221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747dd0d-fa67-4d9f-8667-ca4eb8eb6344 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:15.452616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1121/2234] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:15 2022] PUT /identity/v3/projects/0cea7bd224ee4210b5b2fb4fc742b7e6/users/97fbe5a69283452e9a3da3b25d3156a9/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:15.463146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69901fa-88b6-4b59-868e-a860525b0aa7 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:15.463489 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69901fa-88b6-4b59-868e-a860525b0aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:15.463785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69901fa-88b6-4b59-868e-a860525b0aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:15.497709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b69901fa-88b6-4b59-868e-a860525b0aa7 None None] MFA Rules not processed for user `97fbe5a69283452e9a3da3b25d3156a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:15.499093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b69901fa-88b6-4b59-868e-a860525b0aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:15.601151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1114/2235] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:15.612321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747dd0d-fa67-4d9f-8667-ca4eb8eb6344 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:15.688128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747dd0d-fa67-4d9f-8667-ca4eb8eb6344 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:15.693515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80365fd8-40e8-4374-9806-4c193f4a3fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:15.694173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80365fd8-40e8-4374-9806-4c193f4a3fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:15.694552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80365fd8-40e8-4374-9806-4c193f4a3fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:15.694828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80365fd8-40e8-4374-9806-4c193f4a3fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:15.695478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80365fd8-40e8-4374-9806-4c193f4a3fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:15.697523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80365fd8-40e8-4374-9806-4c193f4a3fdb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:15.794065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1122/2236] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:15 2022] POST /identity/v3/projects => generated 367 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:15.798329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69901fa-88b6-4b59-868e-a860525b0aa7 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:15.809977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69901fa-88b6-4b59-868e-a860525b0aa7 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:15.816446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:15.868826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5959f-3d4e-4bac-9e64-02b37b7f647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:15.869430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5959f-3d4e-4bac-9e64-02b37b7f647a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:15.869773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5959f-3d4e-4bac-9e64-02b37b7f647a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:15.870005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5959f-3d4e-4bac-9e64-02b37b7f647a admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:15.870636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5959f-3d4e-4bac-9e64-02b37b7f647a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:15.873078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5959f-3d4e-4bac-9e64-02b37b7f647a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:16.031330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1115/2237] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:15 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:16.032574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80365fd8-40e8-4374-9806-4c193f4a3fdb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:16.044385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80365fd8-40e8-4374-9806-4c193f4a3fdb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:16.049475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84312805-11e2-4ca4-8223-40e812a197ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:16.050136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84312805-11e2-4ca4-8223-40e812a197ac admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:16.050408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84312805-11e2-4ca4-8223-40e812a197ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:16.050680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84312805-11e2-4ca4-8223-40e812a197ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:16.058365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84312805-11e2-4ca4-8223-40e812a197ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:16.058365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84312805-11e2-4ca4-8223-40e812a197ac admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:16.069606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1123/2238] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:16.074324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5959f-3d4e-4bac-9e64-02b37b7f647a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:16.085798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5959f-3d4e-4bac-9e64-02b37b7f647a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:16.090756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f6a01-3b48-4c4d-8f0a-d4f1f10f8e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:16.091427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f6a01-3b48-4c4d-8f0a-d4f1f10f8e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:16.091710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f6a01-3b48-4c4d-8f0a-d4f1f10f8e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:16.092051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f6a01-3b48-4c4d-8f0a-d4f1f10f8e64 admin admin] PATH_INFO: `/v3/projects/a11e8266b3b241fb95365f89a18650c2/users/1f06dd064b46476fbe5ce679d4beea55/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:16.108827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f6a01-3b48-4c4d-8f0a-d4f1f10f8e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11e8266b3b241fb95365f89a18650c2, user_id=1f06dd064b46476fbe5ce679d4beea55, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:16.113837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f6a01-3b48-4c4d-8f0a-d4f1f10f8e64 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:16.238417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1116/2239] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:16 2022] PUT /identity/v3/projects/a11e8266b3b241fb95365f89a18650c2/users/1f06dd064b46476fbe5ce679d4beea55/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:16.242579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84312805-11e2-4ca4-8223-40e812a197ac admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:16.256478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84312805-11e2-4ca4-8223-40e812a197ac admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:16.344896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e70c61-7bc8-4e82-8074-98afcfaa6adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:16.345831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e70c61-7bc8-4e82-8074-98afcfaa6adb admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:16.346336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e70c61-7bc8-4e82-8074-98afcfaa6adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:16.346744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e70c61-7bc8-4e82-8074-98afcfaa6adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:16.347537 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e70c61-7bc8-4e82-8074-98afcfaa6adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:16.350130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e70c61-7bc8-4e82-8074-98afcfaa6adb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:16.362285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1124/2240] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:16.366768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f6a01-3b48-4c4d-8f0a-d4f1f10f8e64 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:16.378910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f6a01-3b48-4c4d-8f0a-d4f1f10f8e64 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:16.385179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f6e7c-de06-485d-8423-40b92bd9c593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:16.386009 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f6e7c-de06-485d-8423-40b92bd9c593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:16.386302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f6e7c-de06-485d-8423-40b92bd9c593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:16.386667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f6e7c-de06-485d-8423-40b92bd9c593 admin admin] PATH_INFO: `/v3/projects/a11e8266b3b241fb95365f89a18650c2/users/1f06dd064b46476fbe5ce679d4beea55/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:16.388785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f6e7c-de06-485d-8423-40b92bd9c593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11e8266b3b241fb95365f89a18650c2, user_id=1f06dd064b46476fbe5ce679d4beea55, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:16.391525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f6e7c-de06-485d-8423-40b92bd9c593 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:16.402509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1117/2241] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:16 2022] PUT /identity/v3/projects/a11e8266b3b241fb95365f89a18650c2/users/1f06dd064b46476fbe5ce679d4beea55/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:16.413186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53649d-972d-4090-8ae3-c48dc33e560d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:16.413475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53649d-972d-4090-8ae3-c48dc33e560d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:16.413758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53649d-972d-4090-8ae3-c48dc33e560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:16.449491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5d53649d-972d-4090-8ae3-c48dc33e560d None None] MFA Rules not processed for user `1f06dd064b46476fbe5ce679d4beea55`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:16.451012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5d53649d-972d-4090-8ae3-c48dc33e560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:16.533767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1125/2242] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:16.545373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f6e7c-de06-485d-8423-40b92bd9c593 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:16.547239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2d9f6e7c-de06-485d-8423-40b92bd9c593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:16.615471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f6e7c-de06-485d-8423-40b92bd9c593 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:16.619378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396e5c7-5f1c-41bd-82cc-a063e479389e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:16.619998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e5c7-5f1c-41bd-82cc-a063e479389e service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:16.620667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e5c7-5f1c-41bd-82cc-a063e479389e service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:16.621887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e5c7-5f1c-41bd-82cc-a063e479389e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:16.631604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396e5c7-5f1c-41bd-82cc-a063e479389e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:16.633119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396e5c7-5f1c-41bd-82cc-a063e479389e service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:16.648581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1118/2243] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:16.942318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53649d-972d-4090-8ae3-c48dc33e560d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:16.950019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:16.956382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53649d-972d-4090-8ae3-c48dc33e560d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:16.960350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585d272-8693-41cd-9672-08fdedcffd63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:16.961110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585d272-8693-41cd-9672-08fdedcffd63 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:16.961462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585d272-8693-41cd-9672-08fdedcffd63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:16.961783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585d272-8693-41cd-9672-08fdedcffd63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:17.019858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585d272-8693-41cd-9672-08fdedcffd63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:17.021685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585d272-8693-41cd-9672-08fdedcffd63 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:17.031919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1126/2244] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:17.335708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:18.859157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:19.007740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:19.567651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:19.717310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:20.854730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:21.014399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:21.321746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:21.350348 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:21.580323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:21.637885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:21.746447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:21.842481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:21.934698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:21.962895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:22.036404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:22.271645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:22.723530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:22.815080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:23.106294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:23.140293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:23.694770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:23.928984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:23.997608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:27.149360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396e5c7-5f1c-41bd-82cc-a063e479389e service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:27.226865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396e5c7-5f1c-41bd-82cc-a063e479389e service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:27.231749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:27.232723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:27.232723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:27.232912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] PATH_INFO: `/v3/users/97fbe5a69283452e9a3da3b25d3156a9` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:27.234083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] RBAC: Authorizing `identity:delete_user(user_id=97fbe5a69283452e9a3da3b25d3156a9)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:27.237684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:27.434184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97fbe5a69283452e9a3da3b25d3156a9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:27.435163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97fbe5a69283452e9a3da3b25d3156a9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:27.435794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97fbe5a69283452e9a3da3b25d3156a9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:27.498639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97fbe5a69283452e9a3da3b25d3156a9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:27.510547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97fbe5a69283452e9a3da3b25d3156a9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:27.534854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1119/2245] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:27 2022] DELETE /identity/v3/users/97fbe5a69283452e9a3da3b25d3156a9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:27.540466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fa2f0-a64c-4c82-a0f2-07f458fad29d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:27.547271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fa2f0-a64c-4c82-a0f2-07f458fad29d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:27.547271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fa2f0-a64c-4c82-a0f2-07f458fad29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:27.569241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-209fa2f0-a64c-4c82-a0f2-07f458fad29d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:27.570539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-209fa2f0-a64c-4c82-a0f2-07f458fad29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:27.664167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1127/2246] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:27.677761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:27.679970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:27.700095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:27.760295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d14fb-fa4c-4c89-acfa-2587310fa6be admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:27.764639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adb2ff-d0b7-4834-986e-0e3c7b4bc952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:27.765332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adb2ff-d0b7-4834-986e-0e3c7b4bc952 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:27.765684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adb2ff-d0b7-4834-986e-0e3c7b4bc952 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:27.766100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adb2ff-d0b7-4834-986e-0e3c7b4bc952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:27.776314 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adb2ff-d0b7-4834-986e-0e3c7b4bc952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:27.778827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adb2ff-d0b7-4834-986e-0e3c7b4bc952 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:27.796421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1120/2247] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:28.286486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fa2f0-a64c-4c82-a0f2-07f458fad29d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:28.288112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-209fa2f0-a64c-4c82-a0f2-07f458fad29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:28.301727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fa2f0-a64c-4c82-a0f2-07f458fad29d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:28.307807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:28.309686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:28.310014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:28.310870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] PATH_INFO: `/v3/projects/0cea7bd224ee4210b5b2fb4fc742b7e6` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:28.312107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cea7bd224ee4210b5b2fb4fc742b7e6)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:28.314156 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:28.499986 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cea7bd224ee4210b5b2fb4fc742b7e6', 'request_id': 'req-7b51944c-8ebc-4922-965e-7d7b4a790fcc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:28.502039 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:28.506126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cea7bd224ee4210b5b2fb4fc742b7e6', 'request_id': 'req-7b51944c-8ebc-4922-965e-7d7b4a790fcc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:28.608221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cea7bd224ee4210b5b2fb4fc742b7e6', 'request_id': 'req-7b51944c-8ebc-4922-965e-7d7b4a790fcc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:28.621048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] The token cache is being invalidate because project 0cea7bd224ee4210b5b2fb4fc742b7e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:28.621342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:28.622040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:28.623637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1128/2248] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:28 2022] DELETE /identity/v3/projects/0cea7bd224ee4210b5b2fb4fc742b7e6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:28.627743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adb2ff-d0b7-4834-986e-0e3c7b4bc952 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:28.637793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adb2ff-d0b7-4834-986e-0e3c7b4bc952 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:28.704979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:28.705634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:28.705970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:28.706222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] PATH_INFO: `/v3/users/1f06dd064b46476fbe5ce679d4beea55` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:28.707359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f06dd064b46476fbe5ce679d4beea55)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:28.709344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:28.892531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f06dd064b46476fbe5ce679d4beea55'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:28.897587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f06dd064b46476fbe5ce679d4beea55'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:28.897587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f06dd064b46476fbe5ce679d4beea55'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:28.991226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f06dd064b46476fbe5ce679d4beea55'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:28.999695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ab85a3c9-f5b3-4482-8429-1e0a6bb0c23f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f06dd064b46476fbe5ce679d4beea55'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:29.018725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1121/2249] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:28 2022] DELETE /identity/v3/users/1f06dd064b46476fbe5ce679d4beea55 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:29.578684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:29.580221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:29.595763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51944c-8ebc-4922-965e-7d7b4a790fcc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:29.674678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:29.676189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:29.676476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:29.676758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] PATH_INFO: `/v3/projects/a11e8266b3b241fb95365f89a18650c2` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:29.678227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11e8266b3b241fb95365f89a18650c2)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:29.680181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:29.820495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11e8266b3b241fb95365f89a18650c2', 'request_id': 'req-3954c711-7276-4591-ade3-474b3873e515'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:29.826214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11e8266b3b241fb95365f89a18650c2', 'request_id': 'req-3954c711-7276-4591-ade3-474b3873e515'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:29.888960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11e8266b3b241fb95365f89a18650c2', 'request_id': 'req-3954c711-7276-4591-ade3-474b3873e515'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:29.902151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] The token cache is being invalidate because project a11e8266b3b241fb95365f89a18650c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:29.902440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:29.903109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:29.904684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1129/2250] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:29 2022] DELETE /identity/v3/projects/a11e8266b3b241fb95365f89a18650c2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:29.910186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066fe8e-856b-47ab-896b-b72abc8e6180 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:29.910664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066fe8e-856b-47ab-896b-b72abc8e6180 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:29.910664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066fe8e-856b-47ab-896b-b72abc8e6180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:29.935267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8066fe8e-856b-47ab-896b-b72abc8e6180 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:29.936539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8066fe8e-856b-47ab-896b-b72abc8e6180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:29.947524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:29.949066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:29.973083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:30.022464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1122/2251] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:30.025125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d46874-c22c-496c-8e59-1fe26f0ec62b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.025412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d46874-c22c-496c-8e59-1fe26f0ec62b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.025706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d46874-c22c-496c-8e59-1fe26f0ec62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.028812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954c711-7276-4591-ade3-474b3873e515 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:30.037757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:30.038557 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.038835 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.039108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] PATH_INFO: `/v3/users/223cf8031506481c976b3c1a0c424fc7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.040474 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=223cf8031506481c976b3c1a0c424fc7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:30.042602 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:30.052976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e1d46874-c22c-496c-8e59-1fe26f0ec62b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:30.054305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e1d46874-c22c-496c-8e59-1fe26f0ec62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:30.057845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:30.070794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1123/2252] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:30.071740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d46874-c22c-496c-8e59-1fe26f0ec62b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:30.083476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d46874-c22c-496c-8e59-1fe26f0ec62b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:30.088864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7f143-49e9-487d-8d6f-6421853780b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:30.089721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f143-49e9-487d-8d6f-6421853780b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.090108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f143-49e9-487d-8d6f-6421853780b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.090386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f143-49e9-487d-8d6f-6421853780b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.091055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7f143-49e9-487d-8d6f-6421853780b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:30.094933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7f143-49e9-487d-8d6f-6421853780b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:30.094933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7f143-49e9-487d-8d6f-6421853780b2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:30.115389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1124/2253] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:30.119969 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7f143-49e9-487d-8d6f-6421853780b2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:30.134099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7f143-49e9-487d-8d6f-6421853780b2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:30.139763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bde0e-5572-4519-870d-a6369a8216ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:30.140392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bde0e-5572-4519-870d-a6369a8216ac admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.140674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bde0e-5572-4519-870d-a6369a8216ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.140946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bde0e-5572-4519-870d-a6369a8216ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.141715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bde0e-5572-4519-870d-a6369a8216ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:30.143936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bde0e-5572-4519-870d-a6369a8216ac admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:30.204294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '223cf8031506481c976b3c1a0c424fc7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:30.204294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '223cf8031506481c976b3c1a0c424fc7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:30.204294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '223cf8031506481c976b3c1a0c424fc7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:30.211821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '223cf8031506481c976b3c1a0c424fc7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:30.233818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e93c95f0-12c7-4b3e-9c0a-6de30d0ff107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '223cf8031506481c976b3c1a0c424fc7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:30.249915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1125/2254] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:30 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:30.253448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bde0e-5572-4519-870d-a6369a8216ac admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:30.255898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e86bde0e-5572-4519-870d-a6369a8216ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:30.330683 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1130/2255] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:29 2022] DELETE /identity/v3/users/223cf8031506481c976b3c1a0c424fc7 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:30.334677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeaf37-a369-4da2-a1e0-c9041412db37 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.335038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeaf37-a369-4da2-a1e0-c9041412db37 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.335377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeaf37-a369-4da2-a1e0-c9041412db37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.360299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-75eeaf37-a369-4da2-a1e0-c9041412db37 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:30.362045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-75eeaf37-a369-4da2-a1e0-c9041412db37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:30.366770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bde0e-5572-4519-870d-a6369a8216ac admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:30.439073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1264132-9000-45ff-9c49-2470b14e7e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:30.439922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1264132-9000-45ff-9c49-2470b14e7e90 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.440336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1264132-9000-45ff-9c49-2470b14e7e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.440742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1264132-9000-45ff-9c49-2470b14e7e90 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.441503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1264132-9000-45ff-9c49-2470b14e7e90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:30.443476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1264132-9000-45ff-9c49-2470b14e7e90 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:30.446789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1131/2256] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:30.454231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeaf37-a369-4da2-a1e0-c9041412db37 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:30.457160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-75eeaf37-a369-4da2-a1e0-c9041412db37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:30.519138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeaf37-a369-4da2-a1e0-c9041412db37 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:30.522840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213657d-5649-4306-a398-6edbf6b3b6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:30.525352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213657d-5649-4306-a398-6edbf6b3b6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.525352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213657d-5649-4306-a398-6edbf6b3b6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.525352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213657d-5649-4306-a398-6edbf6b3b6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.534732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213657d-5649-4306-a398-6edbf6b3b6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:30.536227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213657d-5649-4306-a398-6edbf6b3b6b3 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:30.547449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1132/2257] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:30.552875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1126/2258] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:30 2022] POST /identity/v3/users => generated 290 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:27:30.557937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213657d-5649-4306-a398-6edbf6b3b6b3 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:30.571052 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213657d-5649-4306-a398-6edbf6b3b6b3 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:30.595897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dcdd4-31d6-4aa7-a327-7983e1990707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:30.597670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dcdd4-31d6-4aa7-a327-7983e1990707 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.597670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dcdd4-31d6-4aa7-a327-7983e1990707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.597670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dcdd4-31d6-4aa7-a327-7983e1990707 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.598249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dcdd4-31d6-4aa7-a327-7983e1990707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:30.600055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dcdd4-31d6-4aa7-a327-7983e1990707 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:30.614823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1133/2259] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:30.618925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1264132-9000-45ff-9c49-2470b14e7e90 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:30.648627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1264132-9000-45ff-9c49-2470b14e7e90 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:30.654109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa58d3-8a78-4c9b-a3e8-0c82764d05ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:30.654976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa58d3-8a78-4c9b-a3e8-0c82764d05ed admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.655436 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa58d3-8a78-4c9b-a3e8-0c82764d05ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.655731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa58d3-8a78-4c9b-a3e8-0c82764d05ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.656400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa58d3-8a78-4c9b-a3e8-0c82764d05ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:30.658068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa58d3-8a78-4c9b-a3e8-0c82764d05ed admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:30.669536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1127/2260] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:30.673718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dcdd4-31d6-4aa7-a327-7983e1990707 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:30.685440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dcdd4-31d6-4aa7-a327-7983e1990707 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:30.691331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876671b-4a7a-49fc-9bcf-5fff17435333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:30.692106 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876671b-4a7a-49fc-9bcf-5fff17435333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.692449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876671b-4a7a-49fc-9bcf-5fff17435333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.692744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876671b-4a7a-49fc-9bcf-5fff17435333 admin admin] PATH_INFO: `/v3/projects/4234cbca1490425c9385e6b45fc6ad9f/users/2b536f748f6c43d98fe16ebe9055ae1c/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.708551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876671b-4a7a-49fc-9bcf-5fff17435333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4234cbca1490425c9385e6b45fc6ad9f, user_id=2b536f748f6c43d98fe16ebe9055ae1c, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:30.711338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876671b-4a7a-49fc-9bcf-5fff17435333 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:30.786901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1134/2261] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:30 2022] PUT /identity/v3/projects/4234cbca1490425c9385e6b45fc6ad9f/users/2b536f748f6c43d98fe16ebe9055ae1c/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:30.798115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38844c1-7130-49ad-90ff-20a80d959a74 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:30.798399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38844c1-7130-49ad-90ff-20a80d959a74 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:30.798675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38844c1-7130-49ad-90ff-20a80d959a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:30.829213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b38844c1-7130-49ad-90ff-20a80d959a74 None None] MFA Rules not processed for user `2b536f748f6c43d98fe16ebe9055ae1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:30.830462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b38844c1-7130-49ad-90ff-20a80d959a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:30.844334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:30.905063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1128/2262] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:30.910350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876671b-4a7a-49fc-9bcf-5fff17435333 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:30.998456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876671b-4a7a-49fc-9bcf-5fff17435333 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.003741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1136dac-6f8d-4ad0-9661-718e5c0eba94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.004354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1136dac-6f8d-4ad0-9661-718e5c0eba94 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.004663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1136dac-6f8d-4ad0-9661-718e5c0eba94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.004935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1136dac-6f8d-4ad0-9661-718e5c0eba94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.005569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1136dac-6f8d-4ad0-9661-718e5c0eba94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.007556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1136dac-6f8d-4ad0-9661-718e5c0eba94 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.038262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38844c1-7130-49ad-90ff-20a80d959a74 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.039769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b38844c1-7130-49ad-90ff-20a80d959a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:31.053966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:31.054148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38844c1-7130-49ad-90ff-20a80d959a74 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.059408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.060044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.060352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.060621 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] PATH_INFO: `/v3/projects/d977d4d8926b4a97877152b8d2326c38` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.061760 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d977d4d8926b4a97877152b8d2326c38)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.063809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.107846 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1135/2263] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:30 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:31.111745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1136dac-6f8d-4ad0-9661-718e5c0eba94 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.160539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd977d4d8926b4a97877152b8d2326c38', 'request_id': 'req-b9124c09-632a-4f96-a55d-289377e40d5d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:31.166714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd977d4d8926b4a97877152b8d2326c38', 'request_id': 'req-b9124c09-632a-4f96-a55d-289377e40d5d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:31.173205 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd977d4d8926b4a97877152b8d2326c38', 'request_id': 'req-b9124c09-632a-4f96-a55d-289377e40d5d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:31.177053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1136dac-6f8d-4ad0-9661-718e5c0eba94 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.238362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ad24f-ec1e-4f39-9dea-d2a9554fe928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.238964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ad24f-ec1e-4f39-9dea-d2a9554fe928 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.239239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ad24f-ec1e-4f39-9dea-d2a9554fe928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.239510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ad24f-ec1e-4f39-9dea-d2a9554fe928 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.240175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ad24f-ec1e-4f39-9dea-d2a9554fe928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.242152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ad24f-ec1e-4f39-9dea-d2a9554fe928 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.257498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] The token cache is being invalidate because project d977d4d8926b4a97877152b8d2326c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:31.257789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:31.258462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9124c09-632a-4f96-a55d-289377e40d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:31.259996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1129/2264] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:31 2022] DELETE /identity/v3/projects/d977d4d8926b4a97877152b8d2326c38 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:31.265572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f120-8d56-4761-8e62-ab4b51fe6f3d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.265798 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f120-8d56-4761-8e62-ab4b51fe6f3d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.266110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f120-8d56-4761-8e62-ab4b51fe6f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.288189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5236f120-8d56-4761-8e62-ab4b51fe6f3d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:31.289411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5236f120-8d56-4761-8e62-ab4b51fe6f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:31.299088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1130/2265] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:31.319498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec03a03-eff7-4205-8937-233e6c688c2d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.319498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec03a03-eff7-4205-8937-233e6c688c2d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.319498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec03a03-eff7-4205-8937-233e6c688c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.325024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1136/2266] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:31 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:27:31.326302 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ad24f-ec1e-4f39-9dea-d2a9554fe928 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.327848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b67ad24f-ec1e-4f39-9dea-d2a9554fe928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:31.332544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2ec03a03-eff7-4205-8937-233e6c688c2d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:31.333733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2ec03a03-eff7-4205-8937-233e6c688c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:31.341722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ad24f-ec1e-4f39-9dea-d2a9554fe928 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.342614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1131/2267] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:31.346619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec03a03-eff7-4205-8937-233e6c688c2d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.371480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ea18e-db7e-4de8-9059-5d02c1708bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.371480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ea18e-db7e-4de8-9059-5d02c1708bfd admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.371480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ea18e-db7e-4de8-9059-5d02c1708bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.371480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ea18e-db7e-4de8-9059-5d02c1708bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.372129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ea18e-db7e-4de8-9059-5d02c1708bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.372129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ea18e-db7e-4de8-9059-5d02c1708bfd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.372435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec03a03-eff7-4205-8937-233e6c688c2d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.372524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1137/2268] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:31.372598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6cf61-77cd-4cbe-b9a1-8db63174a42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.372598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6cf61-77cd-4cbe-b9a1-8db63174a42a admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.372598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6cf61-77cd-4cbe-b9a1-8db63174a42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.372598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6cf61-77cd-4cbe-b9a1-8db63174a42a admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.372860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6cf61-77cd-4cbe-b9a1-8db63174a42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:31.372860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6cf61-77cd-4cbe-b9a1-8db63174a42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.372860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6cf61-77cd-4cbe-b9a1-8db63174a42a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.374591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ea18e-db7e-4de8-9059-5d02c1708bfd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.383807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1132/2269] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:27:31.385124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6cf61-77cd-4cbe-b9a1-8db63174a42a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.385266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ea18e-db7e-4de8-9059-5d02c1708bfd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.390505 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc43cf-1e53-456b-af50-c1c85e77f78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.391162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc43cf-1e53-456b-af50-c1c85e77f78b admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.391439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc43cf-1e53-456b-af50-c1c85e77f78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.391712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc43cf-1e53-456b-af50-c1c85e77f78b admin admin] PATH_INFO: `/v3/projects/e7f610a99ddc4b3f9ab959929af8c6f1/users/f39605e4a1004aac81c29466c6932f57/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.396564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6cf61-77cd-4cbe-b9a1-8db63174a42a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.401699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566cce-3af4-44cf-9bbb-b02c9592c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.402531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566cce-3af4-44cf-9bbb-b02c9592c721 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.411127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566cce-3af4-44cf-9bbb-b02c9592c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.411127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566cce-3af4-44cf-9bbb-b02c9592c721 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.411127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8566cce-3af4-44cf-9bbb-b02c9592c721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.411127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8566cce-3af4-44cf-9bbb-b02c9592c721 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.411725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc43cf-1e53-456b-af50-c1c85e77f78b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f610a99ddc4b3f9ab959929af8c6f1, user_id=f39605e4a1004aac81c29466c6932f57, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.413088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc43cf-1e53-456b-af50-c1c85e77f78b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.546822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1138/2270] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:31 2022] PUT /identity/v3/projects/e7f610a99ddc4b3f9ab959929af8c6f1/users/f39605e4a1004aac81c29466c6932f57/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:31.564333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1133/2271] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:31 2022] POST /identity/v3/projects => generated 371 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:31.592206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc43cf-1e53-456b-af50-c1c85e77f78b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.593128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566cce-3af4-44cf-9bbb-b02c9592c721 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.665988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc43cf-1e53-456b-af50-c1c85e77f78b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.667200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566cce-3af4-44cf-9bbb-b02c9592c721 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.671651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c3911-cd81-4b26-8ad7-2412bcc86d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.672361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c3911-cd81-4b26-8ad7-2412bcc86d35 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.672639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c3911-cd81-4b26-8ad7-2412bcc86d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.672941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afef268-b013-4337-8dbd-211fa63158f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.673056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c3911-cd81-4b26-8ad7-2412bcc86d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.673850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c3911-cd81-4b26-8ad7-2412bcc86d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.674125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afef268-b013-4337-8dbd-211fa63158f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.674406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afef268-b013-4337-8dbd-211fa63158f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.674679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afef268-b013-4337-8dbd-211fa63158f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.675316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afef268-b013-4337-8dbd-211fa63158f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.675508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c3911-cd81-4b26-8ad7-2412bcc86d35 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.677232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afef268-b013-4337-8dbd-211fa63158f5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.685787 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1139/2272] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:27:31.690024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c3911-cd81-4b26-8ad7-2412bcc86d35 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.700599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c3911-cd81-4b26-8ad7-2412bcc86d35 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.706470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78afdb-1bcd-4850-8bdd-fab013fe6e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.707387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78afdb-1bcd-4850-8bdd-fab013fe6e70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.707664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78afdb-1bcd-4850-8bdd-fab013fe6e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.707931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78afdb-1bcd-4850-8bdd-fab013fe6e70 admin admin] PATH_INFO: `/v3/projects/e7f610a99ddc4b3f9ab959929af8c6f1/users/f39605e4a1004aac81c29466c6932f57/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.710274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:31.712632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78afdb-1bcd-4850-8bdd-fab013fe6e70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f610a99ddc4b3f9ab959929af8c6f1, user_id=f39605e4a1004aac81c29466c6932f57, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.713816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78afdb-1bcd-4850-8bdd-fab013fe6e70 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.726879 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1140/2273] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:31 2022] PUT /identity/v3/projects/e7f610a99ddc4b3f9ab959929af8c6f1/users/f39605e4a1004aac81c29466c6932f57/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:31.738548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce20e9a-ab70-4ce4-8a26-ad7129daddfc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.738828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce20e9a-ab70-4ce4-8a26-ad7129daddfc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.739102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce20e9a-ab70-4ce4-8a26-ad7129daddfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.767650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3ce20e9a-ab70-4ce4-8a26-ad7129daddfc None None] MFA Rules not processed for user `f39605e4a1004aac81c29466c6932f57`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:31.768887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3ce20e9a-ab70-4ce4-8a26-ad7129daddfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:31.785517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1134/2274] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:31 2022] POST /identity/v3/users => generated 286 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:27:31.789654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afef268-b013-4337-8dbd-211fa63158f5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.800494 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afef268-b013-4337-8dbd-211fa63158f5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.855040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1141/2275] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:31.865249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e45b39-7a8f-4d84-8722-f6afff924337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.865956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce20e9a-ab70-4ce4-8a26-ad7129daddfc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.866248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e45b39-7a8f-4d84-8722-f6afff924337 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.866635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e45b39-7a8f-4d84-8722-f6afff924337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.867011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e45b39-7a8f-4d84-8722-f6afff924337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.867583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3ce20e9a-ab70-4ce4-8a26-ad7129daddfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:31.867703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e45b39-7a8f-4d84-8722-f6afff924337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.869466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e45b39-7a8f-4d84-8722-f6afff924337 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.882231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1135/2276] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:27:31.886771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e45b39-7a8f-4d84-8722-f6afff924337 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.901096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e45b39-7a8f-4d84-8722-f6afff924337 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.906229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66ebf7-e558-4a4c-b1e2-e55e6583e5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.906847 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66ebf7-e558-4a4c-b1e2-e55e6583e5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.907822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66ebf7-e558-4a4c-b1e2-e55e6583e5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.907822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66ebf7-e558-4a4c-b1e2-e55e6583e5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.908001 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66ebf7-e558-4a4c-b1e2-e55e6583e5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.909893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66ebf7-e558-4a4c-b1e2-e55e6583e5f1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.921926 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1136/2277] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:27:31.926817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66ebf7-e558-4a4c-b1e2-e55e6583e5f1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:31.937339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66ebf7-e558-4a4c-b1e2-e55e6583e5f1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.942938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce20e9a-ab70-4ce4-8a26-ad7129daddfc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:31.943061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ac89-d6fd-45b6-891e-48067b0a77ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.943167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236e082-7a6f-460f-afa9-64a7a1addf7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:31.943289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ac89-d6fd-45b6-891e-48067b0a77ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.943289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ac89-d6fd-45b6-891e-48067b0a77ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.943537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ac89-d6fd-45b6-891e-48067b0a77ab admin admin] PATH_INFO: `/v3/projects/2ac977e5066e413c9e9cc06b5a012fdf/users/b1971caade1b4c34a6c69c600070bc5e/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.944201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236e082-7a6f-460f-afa9-64a7a1addf7f service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:31.944635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236e082-7a6f-460f-afa9-64a7a1addf7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:31.945268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236e082-7a6f-460f-afa9-64a7a1addf7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:31.956348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236e082-7a6f-460f-afa9-64a7a1addf7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.957850 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236e082-7a6f-460f-afa9-64a7a1addf7f service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.965111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ac89-d6fd-45b6-891e-48067b0a77ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac977e5066e413c9e9cc06b5a012fdf, user_id=b1971caade1b4c34a6c69c600070bc5e, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:31.967800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ac89-d6fd-45b6-891e-48067b0a77ab admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:31.968972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1142/2278] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:27:31.991164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:32.048052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1137/2279] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:31 2022] PUT /identity/v3/projects/2ac977e5066e413c9e9cc06b5a012fdf/users/b1971caade1b4c34a6c69c600070bc5e/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:32.053235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052adc74-95aa-4a75-af34-6ebdc79da0b0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:32.053520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052adc74-95aa-4a75-af34-6ebdc79da0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:32.053797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052adc74-95aa-4a75-af34-6ebdc79da0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:32.080101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-052adc74-95aa-4a75-af34-6ebdc79da0b0 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:32.080101 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-052adc74-95aa-4a75-af34-6ebdc79da0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:32.145512 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1143/2280] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:32.156911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ac89-d6fd-45b6-891e-48067b0a77ab admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:32.158626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8111ac89-d6fd-45b6-891e-48067b0a77ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:32.218807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ac89-d6fd-45b6-891e-48067b0a77ab admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:32.222662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7cfeb-81bf-4b0b-9591-6dd744d68ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:32.223405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7cfeb-81bf-4b0b-9591-6dd744d68ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:32.224183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7cfeb-81bf-4b0b-9591-6dd744d68ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:32.224642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7cfeb-81bf-4b0b-9591-6dd744d68ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:32.236388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7cfeb-81bf-4b0b-9591-6dd744d68ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:32.237827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7cfeb-81bf-4b0b-9591-6dd744d68ec8 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:32.242576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:32.249855 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1138/2281] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:32.598724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052adc74-95aa-4a75-af34-6ebdc79da0b0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:32.608832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052adc74-95aa-4a75-af34-6ebdc79da0b0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:32.613547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:32.621482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:32.621482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:32.621482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] PATH_INFO: `/v3/users/2b536f748f6c43d98fe16ebe9055ae1c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:32.621482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b536f748f6c43d98fe16ebe9055ae1c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:32.621482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:32.816470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b536f748f6c43d98fe16ebe9055ae1c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:32.817137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b536f748f6c43d98fe16ebe9055ae1c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:32.817749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b536f748f6c43d98fe16ebe9055ae1c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:32.825668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b536f748f6c43d98fe16ebe9055ae1c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:32.838809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b536f748f6c43d98fe16ebe9055ae1c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:32.945829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1144/2282] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:32 2022] DELETE /identity/v3/users/2b536f748f6c43d98fe16ebe9055ae1c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:32.951000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993c64-c4c4-4fe4-8515-3e8126657826 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:32.951285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993c64-c4c4-4fe4-8515-3e8126657826 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:32.951560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993c64-c4c4-4fe4-8515-3e8126657826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:32.970553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ba993c64-c4c4-4fe4-8515-3e8126657826 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:32.971767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ba993c64-c4c4-4fe4-8515-3e8126657826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:33.010642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:33.038041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1139/2283] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:33.054885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:33.056348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:33.154045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c12d97-1f61-4ea9-8efd-8ae1ec2357fa admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:33.158898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65de006-136e-4255-82a5-674df2208f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:33.159828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65de006-136e-4255-82a5-674df2208f9c service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:33.160154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65de006-136e-4255-82a5-674df2208f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:33.160433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65de006-136e-4255-82a5-674df2208f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:33.174137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65de006-136e-4255-82a5-674df2208f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:33.175717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65de006-136e-4255-82a5-674df2208f9c service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:33.190033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1145/2284] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:27:33.678532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba993c64-c4c4-4fe4-8515-3e8126657826 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:33.686917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ba993c64-c4c4-4fe4-8515-3e8126657826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:33.704192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba993c64-c4c4-4fe4-8515-3e8126657826 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:33.715483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:33.716545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:33.716779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:33.717052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] PATH_INFO: `/v3/projects/4234cbca1490425c9385e6b45fc6ad9f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:33.718289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4234cbca1490425c9385e6b45fc6ad9f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:33.720534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:33.850789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4234cbca1490425c9385e6b45fc6ad9f', 'request_id': 'req-c27baecb-35a1-4e43-9d76-d7a530663289'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:33.863679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4234cbca1490425c9385e6b45fc6ad9f', 'request_id': 'req-c27baecb-35a1-4e43-9d76-d7a530663289'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:33.881255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4234cbca1490425c9385e6b45fc6ad9f', 'request_id': 'req-c27baecb-35a1-4e43-9d76-d7a530663289'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:34.008745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] The token cache is being invalidate because project 4234cbca1490425c9385e6b45fc6ad9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:34.009288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:34.010528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:34.012766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1140/2285] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:33 2022] DELETE /identity/v3/projects/4234cbca1490425c9385e6b45fc6ad9f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:34.016745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65de006-136e-4255-82a5-674df2208f9c service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:34.027142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65de006-136e-4255-82a5-674df2208f9c service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:34.035817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:34.087090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:34.087967 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:34.088374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:34.088654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] PATH_INFO: `/v3/users/f39605e4a1004aac81c29466c6932f57` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:34.090063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39605e4a1004aac81c29466c6932f57)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:34.092086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:34.272736 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39605e4a1004aac81c29466c6932f57'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:34.273573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39605e4a1004aac81c29466c6932f57'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:34.274332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39605e4a1004aac81c29466c6932f57'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:34.285007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39605e4a1004aac81c29466c6932f57'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:34.302276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1aa04e29-0b4c-49fb-ad3b-c0e0f1556bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39605e4a1004aac81c29466c6932f57'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:34.439663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1146/2286] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:34 2022] DELETE /identity/v3/users/f39605e4a1004aac81c29466c6932f57 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:35.011471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:35.013027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:35.027597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27baecb-35a1-4e43-9d76-d7a530663289 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:35.117911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:35.118669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:35.119023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:35.119369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] PATH_INFO: `/v3/projects/e7f610a99ddc4b3f9ab959929af8c6f1` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:35.121391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f610a99ddc4b3f9ab959929af8c6f1)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:35.123660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:35.260246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f610a99ddc4b3f9ab959929af8c6f1', 'request_id': 'req-38dc70f9-b8dc-474d-aea7-00775a2e9d28'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:35.268444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f610a99ddc4b3f9ab959929af8c6f1', 'request_id': 'req-38dc70f9-b8dc-474d-aea7-00775a2e9d28'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:35.277129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f610a99ddc4b3f9ab959929af8c6f1', 'request_id': 'req-38dc70f9-b8dc-474d-aea7-00775a2e9d28'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:35.279434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:35.374923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] The token cache is being invalidate because project e7f610a99ddc4b3f9ab959929af8c6f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:35.375321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:35.376063 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-38dc70f9-b8dc-474d-aea7-00775a2e9d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:35.377961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1141/2287] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:35 2022] DELETE /identity/v3/projects/e7f610a99ddc4b3f9ab959929af8c6f1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:35.382650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae36b4-ae12-45c4-85c6-4bc354c9481e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:35.383783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae36b4-ae12-45c4-85c6-4bc354c9481e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:35.384246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae36b4-ae12-45c4-85c6-4bc354c9481e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:35.410872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-98ae36b4-ae12-45c4-85c6-4bc354c9481e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:35.412684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-98ae36b4-ae12-45c4-85c6-4bc354c9481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:35.487709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1147/2288] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:35.495673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423577c7-1087-42cb-827e-08c4c72b99f9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:35.495962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423577c7-1087-42cb-827e-08c4c72b99f9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:35.496243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423577c7-1087-42cb-827e-08c4c72b99f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:35.521534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-423577c7-1087-42cb-827e-08c4c72b99f9 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:35.522722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-423577c7-1087-42cb-827e-08c4c72b99f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:35.543233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1142/2289] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:35.547573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae36b4-ae12-45c4-85c6-4bc354c9481e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:35.563568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae36b4-ae12-45c4-85c6-4bc354c9481e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:35.571503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a07910-2533-4931-b5d4-f9d61f77f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:35.574047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a07910-2533-4931-b5d4-f9d61f77f713 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:35.574507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a07910-2533-4931-b5d4-f9d61f77f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:35.575284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a07910-2533-4931-b5d4-f9d61f77f713 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:35.576100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a07910-2533-4931-b5d4-f9d61f77f713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:35.576901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a07910-2533-4931-b5d4-f9d61f77f713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:35.579093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a07910-2533-4931-b5d4-f9d61f77f713 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:35.602831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1148/2290] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:35.607833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423577c7-1087-42cb-827e-08c4c72b99f9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:35.618100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423577c7-1087-42cb-827e-08c4c72b99f9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:35.624283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9393e-11f4-4972-a193-4f9e909c11ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:35.624467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9393e-11f4-4972-a193-4f9e909c11ce admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:35.624797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9393e-11f4-4972-a193-4f9e909c11ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:35.625410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9393e-11f4-4972-a193-4f9e909c11ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:35.625832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9393e-11f4-4972-a193-4f9e909c11ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:35.627863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9393e-11f4-4972-a193-4f9e909c11ce admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:35.730932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1143/2291] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:35 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:35.731851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a07910-2533-4931-b5d4-f9d61f77f713 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:35.749716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a07910-2533-4931-b5d4-f9d61f77f713 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:35.768197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:35.827005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c03835-57ef-45b0-84c9-35acab279b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:35.827637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c03835-57ef-45b0-84c9-35acab279b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:35.827924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c03835-57ef-45b0-84c9-35acab279b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:35.828196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c03835-57ef-45b0-84c9-35acab279b33 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:35.835358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c03835-57ef-45b0-84c9-35acab279b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:35.835358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c03835-57ef-45b0-84c9-35acab279b33 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:35.926744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1149/2292] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:35 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:35.928488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9393e-11f4-4972-a193-4f9e909c11ce admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:35.940456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9393e-11f4-4972-a193-4f9e909c11ce admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:35.945329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd5da8-119b-45c8-b0fe-052ace0e6107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:35.945989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd5da8-119b-45c8-b0fe-052ace0e6107 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:35.946274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd5da8-119b-45c8-b0fe-052ace0e6107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:35.946572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd5da8-119b-45c8-b0fe-052ace0e6107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:35.947217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd5da8-119b-45c8-b0fe-052ace0e6107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:35.948967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd5da8-119b-45c8-b0fe-052ace0e6107 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:35.960267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1144/2293] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:27:35.964906 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c03835-57ef-45b0-84c9-35acab279b33 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:35.982055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c03835-57ef-45b0-84c9-35acab279b33 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:35.987449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a0838-c673-4bb4-b433-76634df66825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:35.990040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a0838-c673-4bb4-b433-76634df66825 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:35.990040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a0838-c673-4bb4-b433-76634df66825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:35.990040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a0838-c673-4bb4-b433-76634df66825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:35.990040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a0838-c673-4bb4-b433-76634df66825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:35.991089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a0838-c673-4bb4-b433-76634df66825 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:36.003163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1150/2294] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:36.007311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd5da8-119b-45c8-b0fe-052ace0e6107 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:36.017479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd5da8-119b-45c8-b0fe-052ace0e6107 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:36.023680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327458d-719f-49cd-b82c-259609365209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:36.024673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327458d-719f-49cd-b82c-259609365209 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:36.025166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327458d-719f-49cd-b82c-259609365209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:36.025686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327458d-719f-49cd-b82c-259609365209 admin admin] PATH_INFO: `/v3/projects/f884749043084a55a4d42dad41f653a7/users/7d8c423ed5d3467598c18138d2c3b849/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:36.045906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327458d-719f-49cd-b82c-259609365209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f884749043084a55a4d42dad41f653a7, user_id=7d8c423ed5d3467598c18138d2c3b849, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:36.051215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327458d-719f-49cd-b82c-259609365209 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:36.145610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1145/2295] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:36 2022] PUT /identity/v3/projects/f884749043084a55a4d42dad41f653a7/users/7d8c423ed5d3467598c18138d2c3b849/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:36.156783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f438-78e1-4870-ad06-177f9956170b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:36.157063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f438-78e1-4870-ad06-177f9956170b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:36.157349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f438-78e1-4870-ad06-177f9956170b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:36.195120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b1b6f438-78e1-4870-ad06-177f9956170b None None] MFA Rules not processed for user `7d8c423ed5d3467598c18138d2c3b849`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:36.196860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b1b6f438-78e1-4870-ad06-177f9956170b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:36.293946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1151/2296] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:36.303910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327458d-719f-49cd-b82c-259609365209 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:36.305759 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9327458d-719f-49cd-b82c-259609365209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:36.376704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327458d-719f-49cd-b82c-259609365209 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:36.380751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50194167-5b39-4295-a2ed-b76bc490ef5d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f214c2b77d4cbab2f15122ba21bf38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:36.381363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50194167-5b39-4295-a2ed-b76bc490ef5d service glance] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:36.382159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50194167-5b39-4295-a2ed-b76bc490ef5d service glance] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:36.382159 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50194167-5b39-4295-a2ed-b76bc490ef5d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:36.392618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50194167-5b39-4295-a2ed-b76bc490ef5d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:36.394109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50194167-5b39-4295-a2ed-b76bc490ef5d service glance] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:36.404402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1146/2297] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:27:38.567857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d946e-b946-4ae6-ae36-2a052b3d7605 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:38.568700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d946e-b946-4ae6-ae36-2a052b3d7605 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:38.569121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d946e-b946-4ae6-ae36-2a052b3d7605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:38.623117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6d2d946e-b946-4ae6-ae36-2a052b3d7605 None None] MFA Rules not processed for user `b1971caade1b4c34a6c69c600070bc5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:38.625104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6d2d946e-b946-4ae6-ae36-2a052b3d7605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:38.746059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1152/2298] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:38.750736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50194167-5b39-4295-a2ed-b76bc490ef5d service glance] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:38.752705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-50194167-5b39-4295-a2ed-b76bc490ef5d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:38.848293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50194167-5b39-4295-a2ed-b76bc490ef5d service glance] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:38.854210 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33f35a-539e-4ed1-9059-0f3d79fb05be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:38.855058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33f35a-539e-4ed1-9059-0f3d79fb05be admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:38.855344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33f35a-539e-4ed1-9059-0f3d79fb05be admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:38.855610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33f35a-539e-4ed1-9059-0f3d79fb05be admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:38.856343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33f35a-539e-4ed1-9059-0f3d79fb05be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:38.863010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33f35a-539e-4ed1-9059-0f3d79fb05be admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:38.970799 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1147/2299] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:38 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:38.974703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d946e-b946-4ae6-ae36-2a052b3d7605 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:38.986509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d946e-b946-4ae6-ae36-2a052b3d7605 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:39.000635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:39.056959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154eaae-d405-41ab-a0ce-051a36226e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:39.057614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154eaae-d405-41ab-a0ce-051a36226e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:39.057913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154eaae-d405-41ab-a0ce-051a36226e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:39.058190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154eaae-d405-41ab-a0ce-051a36226e14 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:39.058951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154eaae-d405-41ab-a0ce-051a36226e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:39.060902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154eaae-d405-41ab-a0ce-051a36226e14 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:39.159273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1153/2300] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:38 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:27:39.161317 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33f35a-539e-4ed1-9059-0f3d79fb05be admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:39.174142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33f35a-539e-4ed1-9059-0f3d79fb05be admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:39.180389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e1857-aa8a-4a0a-90c8-cf36d64464ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:39.181127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e1857-aa8a-4a0a-90c8-cf36d64464ec admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:39.181508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e1857-aa8a-4a0a-90c8-cf36d64464ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:39.181903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e1857-aa8a-4a0a-90c8-cf36d64464ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:39.182910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e1857-aa8a-4a0a-90c8-cf36d64464ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:39.184842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e1857-aa8a-4a0a-90c8-cf36d64464ec admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:39.201750 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1148/2301] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:39.202940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154eaae-d405-41ab-a0ce-051a36226e14 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:39.217178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154eaae-d405-41ab-a0ce-051a36226e14 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:39.222993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4484-7be9-4ae4-bf3b-6a1d43449210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:39.223904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4484-7be9-4ae4-bf3b-6a1d43449210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:39.224184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4484-7be9-4ae4-bf3b-6a1d43449210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:39.224461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4484-7be9-4ae4-bf3b-6a1d43449210 admin admin] PATH_INFO: `/v3/projects/7047ba5e9e034ab39ddf41e0fa2224ab/users/f0f583136b2c48eda6308f6ce74b5a92/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:39.241790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc4484-7be9-4ae4-bf3b-6a1d43449210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7047ba5e9e034ab39ddf41e0fa2224ab, user_id=f0f583136b2c48eda6308f6ce74b5a92, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:39.245267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc4484-7be9-4ae4-bf3b-6a1d43449210 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:39.306774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1154/2302] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:39 2022] PUT /identity/v3/projects/7047ba5e9e034ab39ddf41e0fa2224ab/users/f0f583136b2c48eda6308f6ce74b5a92/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:39.311609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e1857-aa8a-4a0a-90c8-cf36d64464ec admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:39.321296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:39.326327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e1857-aa8a-4a0a-90c8-cf36d64464ec admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:39.400920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae395fc-849d-4e52-841c-b007147ef794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:39.401603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae395fc-849d-4e52-841c-b007147ef794 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:39.401909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae395fc-849d-4e52-841c-b007147ef794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:39.402194 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae395fc-849d-4e52-841c-b007147ef794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:39.402857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae395fc-849d-4e52-841c-b007147ef794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:39.404504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae395fc-849d-4e52-841c-b007147ef794 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:39.418431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1149/2303] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:39.423166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4484-7be9-4ae4-bf3b-6a1d43449210 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:39.438081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4484-7be9-4ae4-bf3b-6a1d43449210 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:39.443227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed0207-51ef-405e-ae4b-26e150c86736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:39.443896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed0207-51ef-405e-ae4b-26e150c86736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:39.444307 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed0207-51ef-405e-ae4b-26e150c86736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:39.444579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed0207-51ef-405e-ae4b-26e150c86736 admin admin] PATH_INFO: `/v3/projects/7047ba5e9e034ab39ddf41e0fa2224ab/users/f0f583136b2c48eda6308f6ce74b5a92/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:39.446605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed0207-51ef-405e-ae4b-26e150c86736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7047ba5e9e034ab39ddf41e0fa2224ab, user_id=f0f583136b2c48eda6308f6ce74b5a92, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:39.451220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed0207-51ef-405e-ae4b-26e150c86736 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:39.463628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1155/2304] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:39 2022] PUT /identity/v3/projects/7047ba5e9e034ab39ddf41e0fa2224ab/users/f0f583136b2c48eda6308f6ce74b5a92/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:46.243735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759fdee-09af-4de7-a17f-eed4322a0319 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:46.245675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759fdee-09af-4de7-a17f-eed4322a0319 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:46.245675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759fdee-09af-4de7-a17f-eed4322a0319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:46.278866 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a759fdee-09af-4de7-a17f-eed4322a0319 None None] MFA Rules not processed for user `f0f583136b2c48eda6308f6ce74b5a92`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:46.280085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a759fdee-09af-4de7-a17f-eed4322a0319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:46.447306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f8465-e8cd-40ec-bb70-cb37ecf5f51a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:46.447860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f8465-e8cd-40ec-bb70-cb37ecf5f51a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:46.448039 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f8465-e8cd-40ec-bb70-cb37ecf5f51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:46.452456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1150/2305] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:46.471087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759fdee-09af-4de7-a17f-eed4322a0319 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:46.473589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a759fdee-09af-4de7-a17f-eed4322a0319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:46.489597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-317f8465-e8cd-40ec-bb70-cb37ecf5f51a None None] MFA Rules not processed for user `8a512d59d7b4477b88f7bb9532141d06`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:46.492148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-317f8465-e8cd-40ec-bb70-cb37ecf5f51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:46.638954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1156/2306] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:46.662270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759fdee-09af-4de7-a17f-eed4322a0319 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:46.674088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d5a7e-eee9-4844-9609-7b735812884d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:46.674844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d5a7e-eee9-4844-9609-7b735812884d service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:46.675164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d5a7e-eee9-4844-9609-7b735812884d service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:46.675550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d5a7e-eee9-4844-9609-7b735812884d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:46.702740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:46.750297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d5a7e-eee9-4844-9609-7b735812884d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:46.751924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d5a7e-eee9-4844-9609-7b735812884d service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:46.767064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1151/2307] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:27:49.850765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aef82b-0507-4b38-be87-90bd95bc60fe None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:49.851345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aef82b-0507-4b38-be87-90bd95bc60fe None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:49.851345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aef82b-0507-4b38-be87-90bd95bc60fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:49.877947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-28aef82b-0507-4b38-be87-90bd95bc60fe None None] MFA Rules not processed for user `b1971caade1b4c34a6c69c600070bc5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:49.879413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-28aef82b-0507-4b38-be87-90bd95bc60fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:49.887622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1157/2308] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:27:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:27:56.801430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=17 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:59.867327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=65,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:40.239209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d5a7e-eee9-4844-9609-7b735812884d service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:40.240647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a63d5a7e-eee9-4844-9609-7b735812884d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:40.310487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d5a7e-eee9-4844-9609-7b735812884d service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:40.315729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1f0a4-8efb-42aa-93ce-941fac18fe58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:40.316332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1f0a4-8efb-42aa-93ce-941fac18fe58 service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:40.316611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1f0a4-8efb-42aa-93ce-941fac18fe58 service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:40.316885 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1f0a4-8efb-42aa-93ce-941fac18fe58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:40.318655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c9a1f0a4-8efb-42aa-93ce-941fac18fe58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:40.331431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1f0a4-8efb-42aa-93ce-941fac18fe58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:40.333104 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1f0a4-8efb-42aa-93ce-941fac18fe58 service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:40.345464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1152/2309] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:28:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:28:43.061015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aef82b-0507-4b38-be87-90bd95bc60fe None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:43.130184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aef82b-0507-4b38-be87-90bd95bc60fe None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:43.135824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:43.136544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:43.136813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:43.137088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] PATH_INFO: `/v3/users/7d8c423ed5d3467598c18138d2c3b849` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:43.138471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d8c423ed5d3467598c18138d2c3b849)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:43.140565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:43.376616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d8c423ed5d3467598c18138d2c3b849'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:43.377277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d8c423ed5d3467598c18138d2c3b849'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:43.377889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d8c423ed5d3467598c18138d2c3b849'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:43.385594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d8c423ed5d3467598c18138d2c3b849'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:43.399196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d8c423ed5d3467598c18138d2c3b849'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:43.489372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1158/2310] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:28:43 2022] DELETE /identity/v3/users/7d8c423ed5d3467598c18138d2c3b849 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:43.495150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbca0e9-e0b4-42a3-a002-65a829c11efd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:43.495542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbca0e9-e0b4-42a3-a002-65a829c11efd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:43.495852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbca0e9-e0b4-42a3-a002-65a829c11efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:43.518267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ebbca0e9-e0b4-42a3-a002-65a829c11efd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:43.519503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ebbca0e9-e0b4-42a3-a002-65a829c11efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:43.579320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1153/2311] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:28:43.589371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:43.590904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:43.652924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52c62b-e907-4aa9-a7c9-6e6ae4a78e28 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:43.656794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e48e5d-309c-4020-bcb6-fcf7a32c75c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:43.657399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e48e5d-309c-4020-bcb6-fcf7a32c75c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:43.657681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e48e5d-309c-4020-bcb6-fcf7a32c75c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:43.657988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e48e5d-309c-4020-bcb6-fcf7a32c75c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:43.667461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e48e5d-309c-4020-bcb6-fcf7a32c75c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:43.668917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e48e5d-309c-4020-bcb6-fcf7a32c75c3 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:43.679807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1159/2312] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:28:44.131146 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbca0e9-e0b4-42a3-a002-65a829c11efd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:44.132618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ebbca0e9-e0b4-42a3-a002-65a829c11efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:44.145255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbca0e9-e0b4-42a3-a002-65a829c11efd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:44.150620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:44.151256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:44.151704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:44.151988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] PATH_INFO: `/v3/projects/f884749043084a55a4d42dad41f653a7` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:44.153165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f884749043084a55a4d42dad41f653a7)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:44.158012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:44.299414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f884749043084a55a4d42dad41f653a7', 'request_id': 'req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:44.305953 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f884749043084a55a4d42dad41f653a7', 'request_id': 'req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:44.312438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f884749043084a55a4d42dad41f653a7', 'request_id': 'req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:44.408133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] The token cache is being invalidate because project f884749043084a55a4d42dad41f653a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:28:44.408349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:44.409005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8fee0111-4679-45b7-bd9b-d51d5cdc1cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:44.410558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1154/2313] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:28:44 2022] DELETE /identity/v3/projects/f884749043084a55a4d42dad41f653a7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:44.416228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa7430-ed77-4e79-838c-9f50547b2b98 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:44.416693 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa7430-ed77-4e79-838c-9f50547b2b98 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:44.416822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa7430-ed77-4e79-838c-9f50547b2b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:44.436581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bffa7430-ed77-4e79-838c-9f50547b2b98 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:44.437786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bffa7430-ed77-4e79-838c-9f50547b2b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:44.456874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:44.501211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1160/2314] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:28:44.509398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700840a5-cd66-4c12-8e90-0ec8d5473b82 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:44.509937 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700840a5-cd66-4c12-8e90-0ec8d5473b82 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:44.510034 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700840a5-cd66-4c12-8e90-0ec8d5473b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:44.530652 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-700840a5-cd66-4c12-8e90-0ec8d5473b82 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:44.531887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-700840a5-cd66-4c12-8e90-0ec8d5473b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:44.540489 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1155/2315] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:28:44.544824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa7430-ed77-4e79-838c-9f50547b2b98 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:44.556491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa7430-ed77-4e79-838c-9f50547b2b98 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:44.561350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eed36-944a-4bb5-9066-39828cd3c627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:44.562043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eed36-944a-4bb5-9066-39828cd3c627 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:44.562324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eed36-944a-4bb5-9066-39828cd3c627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:44.562628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eed36-944a-4bb5-9066-39828cd3c627 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:44.563286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376eed36-944a-4bb5-9066-39828cd3c627 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:28:44.563621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376eed36-944a-4bb5-9066-39828cd3c627 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:44.565243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376eed36-944a-4bb5-9066-39828cd3c627 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:44.578895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1161/2316] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:28:44.582787 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700840a5-cd66-4c12-8e90-0ec8d5473b82 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:44.593051 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700840a5-cd66-4c12-8e90-0ec8d5473b82 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:44.605560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeef8a-1acf-4386-bf8e-8b8e2c68f4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:44.606224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeef8a-1acf-4386-bf8e-8b8e2c68f4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:44.606513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeef8a-1acf-4386-bf8e-8b8e2c68f4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:44.606788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeef8a-1acf-4386-bf8e-8b8e2c68f4b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:44.607421 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeef8a-1acf-4386-bf8e-8b8e2c68f4b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:44.609580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeef8a-1acf-4386-bf8e-8b8e2c68f4b6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:44.614624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:44.707966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1156/2317] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:28:44 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:28:44.712696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eed36-944a-4bb5-9066-39828cd3c627 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:44.727239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eed36-944a-4bb5-9066-39828cd3c627 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:44.789277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610c3d2-8371-41a3-a582-e9bd1e150a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:44.790004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610c3d2-8371-41a3-a582-e9bd1e150a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:44.790287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610c3d2-8371-41a3-a582-e9bd1e150a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:44.790562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610c3d2-8371-41a3-a582-e9bd1e150a96 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:44.791253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610c3d2-8371-41a3-a582-e9bd1e150a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:44.793197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610c3d2-8371-41a3-a582-e9bd1e150a96 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:44.912652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1162/2318] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:28:44 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:28:44.916784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeef8a-1acf-4386-bf8e-8b8e2c68f4b6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:44.927657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeef8a-1acf-4386-bf8e-8b8e2c68f4b6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:44.934368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3ee6-90e1-4f6b-b2ad-fede7ed7f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:44.934969 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3ee6-90e1-4f6b-b2ad-fede7ed7f932 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:44.935248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3ee6-90e1-4f6b-b2ad-fede7ed7f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:44.935565 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3ee6-90e1-4f6b-b2ad-fede7ed7f932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:44.936366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e3ee6-90e1-4f6b-b2ad-fede7ed7f932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:44.938064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e3ee6-90e1-4f6b-b2ad-fede7ed7f932 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:44.949028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1157/2319] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:28:44.953162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610c3d2-8371-41a3-a582-e9bd1e150a96 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:44.964163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610c3d2-8371-41a3-a582-e9bd1e150a96 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:44.968214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee7184-a871-4276-ac67-2d1c9e4e041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:44.968805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee7184-a871-4276-ac67-2d1c9e4e041c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:44.969079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee7184-a871-4276-ac67-2d1c9e4e041c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:44.969351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee7184-a871-4276-ac67-2d1c9e4e041c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:44.970017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee7184-a871-4276-ac67-2d1c9e4e041c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:44.971623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee7184-a871-4276-ac67-2d1c9e4e041c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:44.982517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1163/2320] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:28:44.986630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3ee6-90e1-4f6b-b2ad-fede7ed7f932 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:44.996990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3ee6-90e1-4f6b-b2ad-fede7ed7f932 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:45.001527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987212f0-6e2b-4e54-b5c4-d390ca496a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:45.002200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987212f0-6e2b-4e54-b5c4-d390ca496a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:45.002482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987212f0-6e2b-4e54-b5c4-d390ca496a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:45.002759 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987212f0-6e2b-4e54-b5c4-d390ca496a6b admin admin] PATH_INFO: `/v3/projects/2a0acefad93f41ce9782e3f0faf910c9/users/d75685fe89fe436898b921931b747754/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:45.018793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987212f0-6e2b-4e54-b5c4-d390ca496a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a0acefad93f41ce9782e3f0faf910c9, user_id=d75685fe89fe436898b921931b747754, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:45.021459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987212f0-6e2b-4e54-b5c4-d390ca496a6b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:45.130849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1158/2321] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:28:44 2022] PUT /identity/v3/projects/2a0acefad93f41ce9782e3f0faf910c9/users/d75685fe89fe436898b921931b747754/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:45.142020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62233780-833a-4788-af50-53a804ca60f4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:45.142424 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62233780-833a-4788-af50-53a804ca60f4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:45.142698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62233780-833a-4788-af50-53a804ca60f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:45.170590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-62233780-833a-4788-af50-53a804ca60f4 None None] MFA Rules not processed for user `d75685fe89fe436898b921931b747754`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:45.171929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-62233780-833a-4788-af50-53a804ca60f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:45.238642 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1164/2322] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:28:45.250734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987212f0-6e2b-4e54-b5c4-d390ca496a6b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:45.252441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-987212f0-6e2b-4e54-b5c4-d390ca496a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:45.313624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987212f0-6e2b-4e54-b5c4-d390ca496a6b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:45.317818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb98d06-30a7-4485-a763-56c687ad01f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:45.318430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb98d06-30a7-4485-a763-56c687ad01f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:45.318703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb98d06-30a7-4485-a763-56c687ad01f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:45.318977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb98d06-30a7-4485-a763-56c687ad01f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:45.330003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb98d06-30a7-4485-a763-56c687ad01f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:45.331457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb98d06-30a7-4485-a763-56c687ad01f1 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:45.341466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1159/2323] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:28:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:28:45.366998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62233780-833a-4788-af50-53a804ca60f4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:45.386358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:45.430836 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62233780-833a-4788-af50-53a804ca60f4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:45.435520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:45.436167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:45.436529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:45.436807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] PATH_INFO: `/v3/users/d75685fe89fe436898b921931b747754` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:45.437985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d75685fe89fe436898b921931b747754)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:45.439951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:45.623426 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd75685fe89fe436898b921931b747754'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:45.624099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd75685fe89fe436898b921931b747754'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:45.624722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd75685fe89fe436898b921931b747754'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:45.632165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd75685fe89fe436898b921931b747754'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:45.646335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd75685fe89fe436898b921931b747754'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:45.735976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1165/2324] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:28:45 2022] DELETE /identity/v3/users/d75685fe89fe436898b921931b747754 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:45.740485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88ebf5-f42b-456b-8302-b7003adc5c97 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:45.740977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88ebf5-f42b-456b-8302-b7003adc5c97 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:45.741067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88ebf5-f42b-456b-8302-b7003adc5c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:45.761395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9e88ebf5-f42b-456b-8302-b7003adc5c97 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:45.762666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9e88ebf5-f42b-456b-8302-b7003adc5c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:45.823906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1160/2325] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:28:45.833443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:45.834895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:45.893019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ee542-e133-4bec-8ce3-9214581de6cf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:45.896541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461018b-cfaf-46ab-a88d-544a21bb4844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:45.897144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461018b-cfaf-46ab-a88d-544a21bb4844 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:45.897427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461018b-cfaf-46ab-a88d-544a21bb4844 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:45.898234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461018b-cfaf-46ab-a88d-544a21bb4844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:45.907516 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461018b-cfaf-46ab-a88d-544a21bb4844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:45.908965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461018b-cfaf-46ab-a88d-544a21bb4844 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:45.919175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1166/2326] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:28:46.331458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88ebf5-f42b-456b-8302-b7003adc5c97 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:46.335237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9e88ebf5-f42b-456b-8302-b7003adc5c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:46.371667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88ebf5-f42b-456b-8302-b7003adc5c97 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:46.376633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:46.377276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:46.377557 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:46.377833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] PATH_INFO: `/v3/projects/2a0acefad93f41ce9782e3f0faf910c9` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:46.379111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a0acefad93f41ce9782e3f0faf910c9)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:46.381180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:46.512732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:46.514021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a0acefad93f41ce9782e3f0faf910c9', 'request_id': 'req-462b4600-eafe-44c4-b5fd-2d6ea6207342'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:46.519581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a0acefad93f41ce9782e3f0faf910c9', 'request_id': 'req-462b4600-eafe-44c4-b5fd-2d6ea6207342'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:46.525972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a0acefad93f41ce9782e3f0faf910c9', 'request_id': 'req-462b4600-eafe-44c4-b5fd-2d6ea6207342'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:46.609603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] The token cache is being invalidate because project 2a0acefad93f41ce9782e3f0faf910c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:28:46.609923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:46.610560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:46.612086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1161/2327] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:28:46 2022] DELETE /identity/v3/projects/2a0acefad93f41ce9782e3f0faf910c9 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:51.237232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4bed-7f3c-4576-bb13-7455853b5f36 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:51.237726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4bed-7f3c-4576-bb13-7455853b5f36 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:51.237858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4bed-7f3c-4576-bb13-7455853b5f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:51.258846 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d24b4bed-7f3c-4576-bb13-7455853b5f36 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:51.260136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d24b4bed-7f3c-4576-bb13-7455853b5f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:51.301335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:51.324335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1167/2328] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:28:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:28:51.373822 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:51.375320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:51.448613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4600-eafe-44c4-b5fd-2d6ea6207342 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:51.453773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d099f0-a917-439d-860e-808da9d00ba6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:51.454406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d099f0-a917-439d-860e-808da9d00ba6 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:51.454730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d099f0-a917-439d-860e-808da9d00ba6 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:51.454998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d099f0-a917-439d-860e-808da9d00ba6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:51.463255 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d099f0-a917-439d-860e-808da9d00ba6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:51.464915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d099f0-a917-439d-860e-808da9d00ba6 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:51.473074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1162/2329] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:28:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:28:51.670090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4bed-7f3c-4576-bb13-7455853b5f36 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:51.680396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d24b4bed-7f3c-4576-bb13-7455853b5f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:51.790102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4bed-7f3c-4576-bb13-7455853b5f36 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:51.795735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:51.797002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:51.797749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:51.797749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] PATH_INFO: `/v3/users/b43e457ebcfd4ff29b13f3df2fc4e1e7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:51.802788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b43e457ebcfd4ff29b13f3df2fc4e1e7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:51.802788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:52.052041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b43e457ebcfd4ff29b13f3df2fc4e1e7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:52.052942 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b43e457ebcfd4ff29b13f3df2fc4e1e7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:52.053780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b43e457ebcfd4ff29b13f3df2fc4e1e7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:52.062319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b43e457ebcfd4ff29b13f3df2fc4e1e7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:52.077727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b43e457ebcfd4ff29b13f3df2fc4e1e7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:52.151479 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1168/2330] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:28:51 2022] DELETE /identity/v3/users/b43e457ebcfd4ff29b13f3df2fc4e1e7 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:52.158041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9f542-25fd-43f0-84c8-71cbb6e42536 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:52.158524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9f542-25fd-43f0-84c8-71cbb6e42536 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:52.158890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9f542-25fd-43f0-84c8-71cbb6e42536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:52.190194 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-25d9f542-25fd-43f0-84c8-71cbb6e42536 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:52.191870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-25d9f542-25fd-43f0-84c8-71cbb6e42536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:52.290410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1163/2331] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:28:52.298888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:52.301929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:52.416344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c06f6-9019-481f-a53d-8b33a72cf0f0 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:52.424453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63676d95-cc84-4089-9fa9-46b8b5bffe05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:52.425406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63676d95-cc84-4089-9fa9-46b8b5bffe05 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:52.428452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63676d95-cc84-4089-9fa9-46b8b5bffe05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:52.428452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63676d95-cc84-4089-9fa9-46b8b5bffe05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:52.438581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63676d95-cc84-4089-9fa9-46b8b5bffe05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:52.440686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63676d95-cc84-4089-9fa9-46b8b5bffe05 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:52.455655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1169/2332] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:28:53.379660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9f542-25fd-43f0-84c8-71cbb6e42536 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:53.382256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-25d9f542-25fd-43f0-84c8-71cbb6e42536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:53.411806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9f542-25fd-43f0-84c8-71cbb6e42536 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:53.423361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:53.423361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:53.423361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:53.423361 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] PATH_INFO: `/v3/projects/151ecdfda6ca4ee6824d9f4a7d646520` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:53.423763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=151ecdfda6ca4ee6824d9f4a7d646520)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:53.423763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:53.555508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151ecdfda6ca4ee6824d9f4a7d646520', 'request_id': 'req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:53.568307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151ecdfda6ca4ee6824d9f4a7d646520', 'request_id': 'req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:53.572673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:53.580198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151ecdfda6ca4ee6824d9f4a7d646520', 'request_id': 'req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:53.658568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] The token cache is being invalidate because project 151ecdfda6ca4ee6824d9f4a7d646520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:28:53.658568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:53.658568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cba40a5f-4ff3-48b4-8786-2f2f4e1934bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:53.658568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1164/2333] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:28:53 2022] DELETE /identity/v3/projects/151ecdfda6ca4ee6824d9f4a7d646520 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:53.662870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b615dd3-96e3-414a-b089-d57da543e932 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:53.663375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b615dd3-96e3-414a-b089-d57da543e932 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:53.663477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b615dd3-96e3-414a-b089-d57da543e932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:53.678938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:53.685581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9b615dd3-96e3-414a-b089-d57da543e932 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:53.686006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9b615dd3-96e3-414a-b089-d57da543e932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:53.760085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1170/2334] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:28:53.769368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce9ecd-c65f-47c2-be65-38bd57a39060 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:53.778838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce9ecd-c65f-47c2-be65-38bd57a39060 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:53.778838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce9ecd-c65f-47c2-be65-38bd57a39060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:53.794939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e1ce9ecd-c65f-47c2-be65-38bd57a39060 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:53.801704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e1ce9ecd-c65f-47c2-be65-38bd57a39060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:53.823746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1165/2335] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:28:53.825627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b615dd3-96e3-414a-b089-d57da543e932 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:53.836884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b615dd3-96e3-414a-b089-d57da543e932 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:53.842221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320334b-6214-4e5e-accc-258653781682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:53.842846 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320334b-6214-4e5e-accc-258653781682 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:53.843170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320334b-6214-4e5e-accc-258653781682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:53.843462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320334b-6214-4e5e-accc-258653781682 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:53.844255 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320334b-6214-4e5e-accc-258653781682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:28:53.844547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320334b-6214-4e5e-accc-258653781682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:53.846271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320334b-6214-4e5e-accc-258653781682 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:53.863314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1171/2336] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:28:53.867616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce9ecd-c65f-47c2-be65-38bd57a39060 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:53.879711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce9ecd-c65f-47c2-be65-38bd57a39060 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:53.889836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccef508-618d-45f4-9459-c48dbd29ff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:53.890793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccef508-618d-45f4-9459-c48dbd29ff1e admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:53.891397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccef508-618d-45f4-9459-c48dbd29ff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:53.891689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccef508-618d-45f4-9459-c48dbd29ff1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:53.892346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccef508-618d-45f4-9459-c48dbd29ff1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:53.894542 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccef508-618d-45f4-9459-c48dbd29ff1e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:53.995969 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1166/2337] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:28:53 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:28:53.998677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320334b-6214-4e5e-accc-258653781682 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:54.020098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320334b-6214-4e5e-accc-258653781682 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:54.181327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa749b-ce87-4788-9476-f7f41656d446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:54.182917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa749b-ce87-4788-9476-f7f41656d446 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:54.183313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa749b-ce87-4788-9476-f7f41656d446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:54.183853 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa749b-ce87-4788-9476-f7f41656d446 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:54.184652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa749b-ce87-4788-9476-f7f41656d446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:54.187281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa749b-ce87-4788-9476-f7f41656d446 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:54.338709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1172/2338] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:28:53 2022] POST /identity/v3/users => generated 292 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:28:54.352561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccef508-618d-45f4-9459-c48dbd29ff1e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:54.355504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccef508-618d-45f4-9459-c48dbd29ff1e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:54.364071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126adea-93df-43d4-90d0-b46a2ee7928f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:54.365472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126adea-93df-43d4-90d0-b46a2ee7928f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:54.365991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126adea-93df-43d4-90d0-b46a2ee7928f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:54.366523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126adea-93df-43d4-90d0-b46a2ee7928f admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:54.368097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126adea-93df-43d4-90d0-b46a2ee7928f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:54.372086 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126adea-93df-43d4-90d0-b46a2ee7928f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:54.384938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1167/2339] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:28:54.389551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa749b-ce87-4788-9476-f7f41656d446 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:54.406619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa749b-ce87-4788-9476-f7f41656d446 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:54.411340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b33e4-8faf-404c-b7ec-08456e45ee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:54.412109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b33e4-8faf-404c-b7ec-08456e45ee51 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:54.412503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b33e4-8faf-404c-b7ec-08456e45ee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:54.413443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b33e4-8faf-404c-b7ec-08456e45ee51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:54.414291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b33e4-8faf-404c-b7ec-08456e45ee51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:54.416024 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b33e4-8faf-404c-b7ec-08456e45ee51 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:54.429907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1173/2340] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:28:54.433230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126adea-93df-43d4-90d0-b46a2ee7928f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:54.445815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126adea-93df-43d4-90d0-b46a2ee7928f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:54.459154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fac19d-e32a-4b77-9044-866796eedf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:54.459958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fac19d-e32a-4b77-9044-866796eedf95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:54.460235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fac19d-e32a-4b77-9044-866796eedf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:54.460503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fac19d-e32a-4b77-9044-866796eedf95 admin admin] PATH_INFO: `/v3/projects/b424e84a169f4399a448f92b4125e6f4/users/132ea2d0014b408183ff0648aa7a7fc9/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:54.479929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fac19d-e32a-4b77-9044-866796eedf95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b424e84a169f4399a448f92b4125e6f4, user_id=132ea2d0014b408183ff0648aa7a7fc9, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:54.482790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fac19d-e32a-4b77-9044-866796eedf95 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:54.580735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1168/2341] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:28:54 2022] PUT /identity/v3/projects/b424e84a169f4399a448f92b4125e6f4/users/132ea2d0014b408183ff0648aa7a7fc9/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:54.582088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97de338-0be7-43f0-aeae-1a07768123b5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:54.582446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97de338-0be7-43f0-aeae-1a07768123b5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:54.582865 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97de338-0be7-43f0-aeae-1a07768123b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:54.628896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d97de338-0be7-43f0-aeae-1a07768123b5 None None] MFA Rules not processed for user `132ea2d0014b408183ff0648aa7a7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:54.630173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d97de338-0be7-43f0-aeae-1a07768123b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:54.720769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1174/2342] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:28:54.740232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fac19d-e32a-4b77-9044-866796eedf95 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:54.743455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-70fac19d-e32a-4b77-9044-866796eedf95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:54.839865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fac19d-e32a-4b77-9044-866796eedf95 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:54.844610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5456d3-eb66-45c6-96cd-79f42df15a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:54.845364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5456d3-eb66-45c6-96cd-79f42df15a29 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:54.845721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5456d3-eb66-45c6-96cd-79f42df15a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:54.846157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5456d3-eb66-45c6-96cd-79f42df15a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:54.856940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5456d3-eb66-45c6-96cd-79f42df15a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:54.862219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5456d3-eb66-45c6-96cd-79f42df15a29 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:54.884661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1169/2343] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:28:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:28:59.251648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97de338-0be7-43f0-aeae-1a07768123b5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:59.257144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:59.329015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97de338-0be7-43f0-aeae-1a07768123b5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:59.333533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:59.334890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:59.334890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:59.334890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] PATH_INFO: `/v3/users/132ea2d0014b408183ff0648aa7a7fc9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:59.335985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=132ea2d0014b408183ff0648aa7a7fc9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:59.338047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:59.535257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132ea2d0014b408183ff0648aa7a7fc9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.536772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132ea2d0014b408183ff0648aa7a7fc9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.542660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132ea2d0014b408183ff0648aa7a7fc9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.543846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5456d3-eb66-45c6-96cd-79f42df15a29 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:59.545298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9d5456d3-eb66-45c6-96cd-79f42df15a29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:59.559002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132ea2d0014b408183ff0648aa7a7fc9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.561496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5456d3-eb66-45c6-96cd-79f42df15a29 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:59.573663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:59.574346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:59.574707 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:59.576412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] PATH_INFO: `/v3/users/3910367683114ce6a7e065fc039efc04` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:59.577941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-400db3e2-6008-4797-a3f0-1a7857117ca6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132ea2d0014b408183ff0648aa7a7fc9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.579410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3910367683114ce6a7e065fc039efc04)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:59.582872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:59.673218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1175/2344] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:28:59 2022] DELETE /identity/v3/users/132ea2d0014b408183ff0648aa7a7fc9 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:59.678472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6011cc9-31d8-4151-94b2-fadc79c941c7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:59.679064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6011cc9-31d8-4151-94b2-fadc79c941c7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:59.679493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6011cc9-31d8-4151-94b2-fadc79c941c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:59.701956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f6011cc9-31d8-4151-94b2-fadc79c941c7 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:59.703425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f6011cc9-31d8-4151-94b2-fadc79c941c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:59.799045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3910367683114ce6a7e065fc039efc04'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.800089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1176/2345] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:28:59.806081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3910367683114ce6a7e065fc039efc04'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.807609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3910367683114ce6a7e065fc039efc04'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.812689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6011cc9-31d8-4151-94b2-fadc79c941c7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:28:59.814086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f6011cc9-31d8-4151-94b2-fadc79c941c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:59.845205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:59.877395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3910367683114ce6a7e065fc039efc04'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.887500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6011cc9-31d8-4151-94b2-fadc79c941c7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:28:59.892845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-64d519c1-e748-4c56-bbf3-c619e10f5c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3910367683114ce6a7e065fc039efc04'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:28:59.893095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:28:59.894284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:59.894430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:59.894841 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:59.897380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:28:59.915281 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:28:59.917128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:28:59.920964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:59.925686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1170/2346] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:28:59 2022] DELETE /identity/v3/users/3910367683114ce6a7e065fc039efc04 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:59.927708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1177/2347] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:28:59.930527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4816d6-bcaa-47fc-a095-c85834d8fb21 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:28:59.930917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4816d6-bcaa-47fc-a095-c85834d8fb21 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:28:59.931298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4816d6-bcaa-47fc-a095-c85834d8fb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:28:59.957768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6d4816d6-bcaa-47fc-a095-c85834d8fb21 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:28:59.959280 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6d4816d6-bcaa-47fc-a095-c85834d8fb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:00.082338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1171/2348] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:00.095946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:00.098547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:00.110840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b321ec-e856-4baf-8693-42c9f988a97d service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:00.243974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fa440-fba0-4ae9-9a40-efd55d731f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:00.245092 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fa440-fba0-4ae9-9a40-efd55d731f9d service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:00.245370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fa440-fba0-4ae9-9a40-efd55d731f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:00.245639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fa440-fba0-4ae9-9a40-efd55d731f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:00.279587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fa440-fba0-4ae9-9a40-efd55d731f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:00.279587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fa440-fba0-4ae9-9a40-efd55d731f9d service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:00.306564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1178/2349] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:00.606423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4816d6-bcaa-47fc-a095-c85834d8fb21 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:00.607883 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6d4816d6-bcaa-47fc-a095-c85834d8fb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:00.630429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4816d6-bcaa-47fc-a095-c85834d8fb21 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:00.654908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:00.657962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:00.658256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:00.660674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] PATH_INFO: `/v3/projects/b424e84a169f4399a448f92b4125e6f4` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:00.667730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b424e84a169f4399a448f92b4125e6f4)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:00.667730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:00.878484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b424e84a169f4399a448f92b4125e6f4', 'request_id': 'req-3ad84a76-6244-4490-9390-f5469f032d28'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:00.908411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b424e84a169f4399a448f92b4125e6f4', 'request_id': 'req-3ad84a76-6244-4490-9390-f5469f032d28'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:00.930920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b424e84a169f4399a448f92b4125e6f4', 'request_id': 'req-3ad84a76-6244-4490-9390-f5469f032d28'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:01.009949 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fa440-fba0-4ae9-9a40-efd55d731f9d service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:01.011430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a84fa440-fba0-4ae9-9a40-efd55d731f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:01.036414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fa440-fba0-4ae9-9a40-efd55d731f9d service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:01.045546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] The token cache is being invalidate because project b424e84a169f4399a448f92b4125e6f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:29:01.046638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:01.049988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3ad84a76-6244-4490-9390-f5469f032d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:01.054844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1172/2350] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:29:00 2022] DELETE /identity/v3/projects/b424e84a169f4399a448f92b4125e6f4 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:01.173082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:01.173082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:01.173082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:01.173082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] PATH_INFO: `/v3/projects/594f9b805c3748f0850f0c7be6524b44` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:01.174109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=594f9b805c3748f0850f0c7be6524b44)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:01.176925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:01.407343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594f9b805c3748f0850f0c7be6524b44', 'request_id': 'req-21c72906-d30b-40a6-98f6-b7655f6976e6'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:01.431746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594f9b805c3748f0850f0c7be6524b44', 'request_id': 'req-21c72906-d30b-40a6-98f6-b7655f6976e6'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:01.588888 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594f9b805c3748f0850f0c7be6524b44', 'request_id': 'req-21c72906-d30b-40a6-98f6-b7655f6976e6'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:01.599838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:01.606378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] The token cache is being invalidate because project 594f9b805c3748f0850f0c7be6524b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:29:01.606678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:01.607531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-21c72906-d30b-40a6-98f6-b7655f6976e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:01.609927 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1179/2351] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:29:01 2022] DELETE /identity/v3/projects/594f9b805c3748f0850f0c7be6524b44 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:01.615526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486ff7a-4f74-4809-a668-986175c8f484 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:01.615526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486ff7a-4f74-4809-a668-986175c8f484 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:01.620038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486ff7a-4f74-4809-a668-986175c8f484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:01.639456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c486ff7a-4f74-4809-a668-986175c8f484 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:01.640783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c486ff7a-4f74-4809-a668-986175c8f484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:01.730607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60729c1b-21f9-438e-8632-ba13b8b8fd39 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:01.730979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60729c1b-21f9-438e-8632-ba13b8b8fd39 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:01.731264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60729c1b-21f9-438e-8632-ba13b8b8fd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:01.735407 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1173/2352] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:01.753555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-60729c1b-21f9-438e-8632-ba13b8b8fd39 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:01.755006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-60729c1b-21f9-438e-8632-ba13b8b8fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:01.769366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486ff7a-4f74-4809-a668-986175c8f484 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:01.774587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1180/2353] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:01.781100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486ff7a-4f74-4809-a668-986175c8f484 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:01.786416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe131aac-21c6-4297-9ea3-e4e75fe3b7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:01.787037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe131aac-21c6-4297-9ea3-e4e75fe3b7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:01.787306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe131aac-21c6-4297-9ea3-e4e75fe3b7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:01.788109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe131aac-21c6-4297-9ea3-e4e75fe3b7c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:01.788831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe131aac-21c6-4297-9ea3-e4e75fe3b7c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:29:01.789139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe131aac-21c6-4297-9ea3-e4e75fe3b7c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:01.791307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe131aac-21c6-4297-9ea3-e4e75fe3b7c2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:01.806325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1174/2354] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:29:01.810797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60729c1b-21f9-438e-8632-ba13b8b8fd39 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:01.823313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60729c1b-21f9-438e-8632-ba13b8b8fd39 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:01.828395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b95af-d4db-401a-97e5-c8beb794eb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:01.830002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b95af-d4db-401a-97e5-c8beb794eb0e admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:01.830002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b95af-d4db-401a-97e5-c8beb794eb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:01.830002 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b95af-d4db-401a-97e5-c8beb794eb0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:01.830870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b95af-d4db-401a-97e5-c8beb794eb0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:01.833060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b95af-d4db-401a-97e5-c8beb794eb0e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:01.923509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1181/2355] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:29:01 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:29:01.927665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe131aac-21c6-4297-9ea3-e4e75fe3b7c2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:01.941662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe131aac-21c6-4297-9ea3-e4e75fe3b7c2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:01.954513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:02.017656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11431dc8-2116-4455-ac9f-c3ae7383ce3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:02.018445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11431dc8-2116-4455-ac9f-c3ae7383ce3f admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:02.018794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11431dc8-2116-4455-ac9f-c3ae7383ce3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:02.019243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11431dc8-2116-4455-ac9f-c3ae7383ce3f admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:02.020180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11431dc8-2116-4455-ac9f-c3ae7383ce3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:02.022739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11431dc8-2116-4455-ac9f-c3ae7383ce3f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:02.094602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1175/2356] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:29:01 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:29:02.098258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b95af-d4db-401a-97e5-c8beb794eb0e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:02.121129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b95af-d4db-401a-97e5-c8beb794eb0e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:02.140252 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd92a88-a6e3-4b81-8e36-691be3f7eff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:02.142462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd92a88-a6e3-4b81-8e36-691be3f7eff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:02.144296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd92a88-a6e3-4b81-8e36-691be3f7eff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:02.144737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd92a88-a6e3-4b81-8e36-691be3f7eff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:02.145695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd92a88-a6e3-4b81-8e36-691be3f7eff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:02.150980 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd92a88-a6e3-4b81-8e36-691be3f7eff9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:02.174513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1182/2357] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:29:02.176198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11431dc8-2116-4455-ac9f-c3ae7383ce3f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:02.191541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11431dc8-2116-4455-ac9f-c3ae7383ce3f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:02.197385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54129be4-218e-47ce-91f8-9a75dea05b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:02.198479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54129be4-218e-47ce-91f8-9a75dea05b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:02.198915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54129be4-218e-47ce-91f8-9a75dea05b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:02.199845 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54129be4-218e-47ce-91f8-9a75dea05b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:02.200674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54129be4-218e-47ce-91f8-9a75dea05b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:02.202512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54129be4-218e-47ce-91f8-9a75dea05b25 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:02.221991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1176/2358] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:29:02.222546 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd92a88-a6e3-4b81-8e36-691be3f7eff9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:02.235475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd92a88-a6e3-4b81-8e36-691be3f7eff9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:02.242491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171fbb-d492-4aeb-9b8a-a15895de9db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:02.243130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171fbb-d492-4aeb-9b8a-a15895de9db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:02.243403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171fbb-d492-4aeb-9b8a-a15895de9db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:02.243685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171fbb-d492-4aeb-9b8a-a15895de9db1 admin admin] PATH_INFO: `/v3/projects/7e02d1127ce24c20822ab10c78edb00b/users/413dcd64b46f4e10947d9347dff434cc/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:02.262885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5171fbb-d492-4aeb-9b8a-a15895de9db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e02d1127ce24c20822ab10c78edb00b, user_id=413dcd64b46f4e10947d9347dff434cc, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:02.265587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5171fbb-d492-4aeb-9b8a-a15895de9db1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:02.362496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc4076-5b62-49f1-b1a9-181644bc2a0a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:02.362496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc4076-5b62-49f1-b1a9-181644bc2a0a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:02.362496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc4076-5b62-49f1-b1a9-181644bc2a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:02.366020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1183/2359] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:29:02 2022] PUT /identity/v3/projects/7e02d1127ce24c20822ab10c78edb00b/users/413dcd64b46f4e10947d9347dff434cc/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:29:02.403974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-16fc4076-5b62-49f1-b1a9-181644bc2a0a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:02.405978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-16fc4076-5b62-49f1-b1a9-181644bc2a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:02.542492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1177/2360] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:29:02.549975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171fbb-d492-4aeb-9b8a-a15895de9db1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:02.551884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a5171fbb-d492-4aeb-9b8a-a15895de9db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:02.650648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171fbb-d492-4aeb-9b8a-a15895de9db1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:02.656115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c88a37-8983-40f7-b9c7-dbd5c9dfe63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:02.657065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c88a37-8983-40f7-b9c7-dbd5c9dfe63a service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:02.657362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c88a37-8983-40f7-b9c7-dbd5c9dfe63a service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:02.657686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c88a37-8983-40f7-b9c7-dbd5c9dfe63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:02.672864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c88a37-8983-40f7-b9c7-dbd5c9dfe63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:02.674425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c88a37-8983-40f7-b9c7-dbd5c9dfe63a service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:02.691383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1184/2361] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:29:09.821281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aba774-a318-4cec-8139-44ab13b9505c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:09.822198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aba774-a318-4cec-8139-44ab13b9505c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:09.822198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aba774-a318-4cec-8139-44ab13b9505c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:09.864368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d4aba774-a318-4cec-8139-44ab13b9505c None None] MFA Rules not processed for user `57be68e76fc14d2ca309571e284509bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:09.866944 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d4aba774-a318-4cec-8139-44ab13b9505c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:10.005313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1178/2362] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:11.862464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9588c01-2813-4e18-84b4-c532b45850f5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:11.863453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9588c01-2813-4e18-84b4-c532b45850f5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:11.863935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9588c01-2813-4e18-84b4-c532b45850f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:11.967640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b9588c01-2813-4e18-84b4-c532b45850f5 None None] MFA Rules not processed for user `413dcd64b46f4e10947d9347dff434cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:11.972871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b9588c01-2813-4e18-84b4-c532b45850f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:12.131586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1185/2363] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:12.140289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aba774-a318-4cec-8139-44ab13b9505c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:12.167655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aba774-a318-4cec-8139-44ab13b9505c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:12.175165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d93cbd-831f-46cd-9566-eeeb08391410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:12.176575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d93cbd-831f-46cd-9566-eeeb08391410 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:12.177996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d93cbd-831f-46cd-9566-eeeb08391410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:12.178546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d93cbd-831f-46cd-9566-eeeb08391410 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:12.179706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d93cbd-831f-46cd-9566-eeeb08391410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:12.182233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d93cbd-831f-46cd-9566-eeeb08391410 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:12.337367 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1179/2364] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:29:12 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:29:12.338602 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9588c01-2813-4e18-84b4-c532b45850f5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:12.354473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9588c01-2813-4e18-84b4-c532b45850f5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:12.386207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:12.461674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b42b37-cb68-432d-8fd0-d19bc0f40403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:12.462487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b42b37-cb68-432d-8fd0-d19bc0f40403 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:12.462870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b42b37-cb68-432d-8fd0-d19bc0f40403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:12.463246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b42b37-cb68-432d-8fd0-d19bc0f40403 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:12.464019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b42b37-cb68-432d-8fd0-d19bc0f40403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:12.466123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b42b37-cb68-432d-8fd0-d19bc0f40403 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:12.633611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1186/2365] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:29:12 2022] POST /identity/v3/users => generated 299 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:29:12.637824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d93cbd-831f-46cd-9566-eeeb08391410 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:12.650804 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d93cbd-831f-46cd-9566-eeeb08391410 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:12.656157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba7835-7a63-463f-873c-de61933273c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:12.656773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba7835-7a63-463f-873c-de61933273c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:12.657057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba7835-7a63-463f-873c-de61933273c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:12.657325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba7835-7a63-463f-873c-de61933273c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:12.658018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba7835-7a63-463f-873c-de61933273c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:12.659866 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba7835-7a63-463f-873c-de61933273c4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:12.671292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1180/2366] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:29:12.675189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b42b37-cb68-432d-8fd0-d19bc0f40403 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:12.688533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b42b37-cb68-432d-8fd0-d19bc0f40403 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:12.711917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee149d-1aec-4d8c-9cd8-4767509dfd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:12.712904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee149d-1aec-4d8c-9cd8-4767509dfd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:12.713344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee149d-1aec-4d8c-9cd8-4767509dfd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:12.713777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee149d-1aec-4d8c-9cd8-4767509dfd91 admin admin] PATH_INFO: `/v3/projects/12ee7a05cead40078bd518b096cc7023/users/1518cd7f166b474d8d8f6557c02d4f75/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:12.735897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee149d-1aec-4d8c-9cd8-4767509dfd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ee7a05cead40078bd518b096cc7023, user_id=1518cd7f166b474d8d8f6557c02d4f75, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:12.740326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee149d-1aec-4d8c-9cd8-4767509dfd91 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:12.853910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1187/2367] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:29:12 2022] PUT /identity/v3/projects/12ee7a05cead40078bd518b096cc7023/users/1518cd7f166b474d8d8f6557c02d4f75/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:12.858429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba7835-7a63-463f-873c-de61933273c4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:12.864776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:12.872234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba7835-7a63-463f-873c-de61933273c4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:12.972644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f21495-cf7d-4993-95f4-f51019aa20e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:12.973797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f21495-cf7d-4993-95f4-f51019aa20e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:12.974368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f21495-cf7d-4993-95f4-f51019aa20e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:12.974836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f21495-cf7d-4993-95f4-f51019aa20e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:12.975694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f21495-cf7d-4993-95f4-f51019aa20e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:12.977518 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f21495-cf7d-4993-95f4-f51019aa20e3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:12.997522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee149d-1aec-4d8c-9cd8-4767509dfd91 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:13.005135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1181/2368] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:29:13.013838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee149d-1aec-4d8c-9cd8-4767509dfd91 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:13.021248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5ac25-37c3-424f-9e2b-86de83609c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:13.022303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5ac25-37c3-424f-9e2b-86de83609c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:13.022691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5ac25-37c3-424f-9e2b-86de83609c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:13.023186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5ac25-37c3-424f-9e2b-86de83609c1d admin admin] PATH_INFO: `/v3/projects/12ee7a05cead40078bd518b096cc7023/users/1518cd7f166b474d8d8f6557c02d4f75/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:13.025714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5ac25-37c3-424f-9e2b-86de83609c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ee7a05cead40078bd518b096cc7023, user_id=1518cd7f166b474d8d8f6557c02d4f75, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:13.029329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5ac25-37c3-424f-9e2b-86de83609c1d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:13.042189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1188/2369] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:29:12 2022] PUT /identity/v3/projects/12ee7a05cead40078bd518b096cc7023/users/1518cd7f166b474d8d8f6557c02d4f75/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:29:17.065728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abeb5af-68be-4aa0-8ce8-da5491fa78e6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:17.072043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abeb5af-68be-4aa0-8ce8-da5491fa78e6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:17.072043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abeb5af-68be-4aa0-8ce8-da5491fa78e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:17.088579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8abeb5af-68be-4aa0-8ce8-da5491fa78e6 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:17.089997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8abeb5af-68be-4aa0-8ce8-da5491fa78e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:17.150694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1182/2370] 10.222.0.83 () {58 vars in 967 bytes} [Wed Aug 24 11:29:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:17.165691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5ac25-37c3-424f-9e2b-86de83609c1d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:17.168240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-49d5ac25-37c3-424f-9e2b-86de83609c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:17.358356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5ac25-37c3-424f-9e2b-86de83609c1d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:17.375542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d99b88-c8bf-4272-9fcc-3b7d2e7e9736 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:17.376709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d99b88-c8bf-4272-9fcc-3b7d2e7e9736 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:17.377151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d99b88-c8bf-4272-9fcc-3b7d2e7e9736 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:17.377605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d99b88-c8bf-4272-9fcc-3b7d2e7e9736 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:17.391158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d99b88-c8bf-4272-9fcc-3b7d2e7e9736 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:17.394168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d99b88-c8bf-4272-9fcc-3b7d2e7e9736 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:17.409530 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1189/2371] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:29:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:29:20.539291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab832db-d0a1-473b-a233-c59b4eab18e6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:20.539762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab832db-d0a1-473b-a233-c59b4eab18e6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:20.539762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab832db-d0a1-473b-a233-c59b4eab18e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:20.570381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1ab832db-d0a1-473b-a233-c59b4eab18e6 None None] MFA Rules not processed for user `1518cd7f166b474d8d8f6557c02d4f75`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:20.574455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1ab832db-d0a1-473b-a233-c59b4eab18e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:20.649010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1183/2372] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:20.668414 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d99b88-c8bf-4272-9fcc-3b7d2e7e9736 service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:20.670685 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e2d99b88-c8bf-4272-9fcc-3b7d2e7e9736 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:20.742041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d99b88-c8bf-4272-9fcc-3b7d2e7e9736 service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:20.746244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ff355-2dd7-44d2-bdf6-44434abfee19 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f214c2b77d4cbab2f15122ba21bf38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:20.746863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ff355-2dd7-44d2-bdf6-44434abfee19 service glance] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:20.747148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ff355-2dd7-44d2-bdf6-44434abfee19 service glance] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:20.747427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ff355-2dd7-44d2-bdf6-44434abfee19 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:20.779783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:20.831814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ff355-2dd7-44d2-bdf6-44434abfee19 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:20.833387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ff355-2dd7-44d2-bdf6-44434abfee19 service glance] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:20.843355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1190/2373] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:21.130947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47916854-8e5f-4870-ab42-27030fa3dbd9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:21.131145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47916854-8e5f-4870-ab42-27030fa3dbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:21.131413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47916854-8e5f-4870-ab42-27030fa3dbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:21.153389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-47916854-8e5f-4870-ab42-27030fa3dbd9 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:21.154679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-47916854-8e5f-4870-ab42-27030fa3dbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:21.168654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1184/2374] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:29:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:21.178011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ff355-2dd7-44d2-bdf6-44434abfee19 service glance] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:21.192198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ff355-2dd7-44d2-bdf6-44434abfee19 service glance] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:21.198370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31ef0f-2c3e-418d-9ce9-012123c46fb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:21.199523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31ef0f-2c3e-418d-9ce9-012123c46fb6 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:21.199523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31ef0f-2c3e-418d-9ce9-012123c46fb6 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:21.199523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31ef0f-2c3e-418d-9ce9-012123c46fb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:21.210021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31ef0f-2c3e-418d-9ce9-012123c46fb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:21.211940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31ef0f-2c3e-418d-9ce9-012123c46fb6 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:21.222936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1191/2375] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:29:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:29:21.259581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb2b21-ed04-4a50-a4e1-bfdab09c7550 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:21.260124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb2b21-ed04-4a50-a4e1-bfdab09c7550 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:21.260574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb2b21-ed04-4a50-a4e1-bfdab09c7550 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:21.262052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1185/2376] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:29:21.276970 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93585f5e-0392-427f-a0c2-8fa4f5019154 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:21.277260 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93585f5e-0392-427f-a0c2-8fa4f5019154 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:21.277607 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93585f5e-0392-427f-a0c2-8fa4f5019154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:21.327701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-93585f5e-0392-427f-a0c2-8fa4f5019154 None None] MFA Rules not processed for user `eb76ad667a744126ab00b38c346e9017`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:21.328944 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-93585f5e-0392-427f-a0c2-8fa4f5019154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:21.435923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb2b21-ed04-4a50-a4e1-bfdab09c7550 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:21.439013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1bfb2b21-ed04-4a50-a4e1-bfdab09c7550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:21.490883 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1192/2377] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:29:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:21.498515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:21.508131 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93585f5e-0392-427f-a0c2-8fa4f5019154 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:21.510102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-93585f5e-0392-427f-a0c2-8fa4f5019154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:21.540122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93585f5e-0392-427f-a0c2-8fa4f5019154 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:21.555891 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64dfeb-2eda-41c7-af42-3ed557324401 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:21.556504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64dfeb-2eda-41c7-af42-3ed557324401 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:21.556789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64dfeb-2eda-41c7-af42-3ed557324401 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:21.557063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64dfeb-2eda-41c7-af42-3ed557324401 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:21.582309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64dfeb-2eda-41c7-af42-3ed557324401 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:21.583109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64dfeb-2eda-41c7-af42-3ed557324401 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:21.612441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1193/2378] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:21.648658 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb2b21-ed04-4a50-a4e1-bfdab09c7550 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:21.662223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:21.663211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:21.663496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:21.663772 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] PATH_INFO: `/v3/users/8a512d59d7b4477b88f7bb9532141d06` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:21.665100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a512d59d7b4477b88f7bb9532141d06)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:21.667295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:21.967490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a512d59d7b4477b88f7bb9532141d06'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:21.968612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a512d59d7b4477b88f7bb9532141d06'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:21.969464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a512d59d7b4477b88f7bb9532141d06'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:22.054320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a512d59d7b4477b88f7bb9532141d06'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:22.063441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c78dd48-a413-40f7-9d13-962531d88220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a512d59d7b4477b88f7bb9532141d06'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:22.089752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1186/2379] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:29:21 2022] DELETE /identity/v3/users/8a512d59d7b4477b88f7bb9532141d06 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:29:22.415828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64dfeb-2eda-41c7-af42-3ed557324401 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:22.444676 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64dfeb-2eda-41c7-af42-3ed557324401 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:22.590209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:22.591964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:22.591964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:22.591964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] PATH_INFO: `/v3/projects/a1b8e9dbce62475eaa97dbad91fcd910` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:22.593298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b8e9dbce62475eaa97dbad91fcd910)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:22.595833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:22.760147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:22.766958 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b8e9dbce62475eaa97dbad91fcd910', 'request_id': 'req-b5ac6949-d307-4224-b544-9c1f26ddda49'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:22.772191 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b8e9dbce62475eaa97dbad91fcd910', 'request_id': 'req-b5ac6949-d307-4224-b544-9c1f26ddda49'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:22.968926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b8e9dbce62475eaa97dbad91fcd910', 'request_id': 'req-b5ac6949-d307-4224-b544-9c1f26ddda49'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:22.990136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] The token cache is being invalidate because project a1b8e9dbce62475eaa97dbad91fcd910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:29:22.990431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:22.991712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b5ac6949-d307-4224-b544-9c1f26ddda49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:22.993581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1194/2380] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:29:22 2022] DELETE /identity/v3/projects/a1b8e9dbce62475eaa97dbad91fcd910 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:22.998647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac938e-59e1-486e-91c9-4c4d57f39e04 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:22.999139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac938e-59e1-486e-91c9-4c4d57f39e04 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:22.999258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac938e-59e1-486e-91c9-4c4d57f39e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:23.026072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5dac938e-59e1-486e-91c9-4c4d57f39e04 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:23.027385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5dac938e-59e1-486e-91c9-4c4d57f39e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:23.129701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1187/2381] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:23.138098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd46b-1d3b-48ae-9691-e76074339523 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:23.139028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd46b-1d3b-48ae-9691-e76074339523 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:23.139528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd46b-1d3b-48ae-9691-e76074339523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:23.177437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-544fd46b-1d3b-48ae-9691-e76074339523 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:23.177437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-544fd46b-1d3b-48ae-9691-e76074339523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:23.190405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1195/2382] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:23.195667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac938e-59e1-486e-91c9-4c4d57f39e04 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:23.209275 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac938e-59e1-486e-91c9-4c4d57f39e04 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:23.212220 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156662d-c056-4228-8096-94e4461993e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:23.215626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156662d-c056-4228-8096-94e4461993e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:23.217586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156662d-c056-4228-8096-94e4461993e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:23.217977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156662d-c056-4228-8096-94e4461993e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:23.219668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156662d-c056-4228-8096-94e4461993e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:29:23.219981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156662d-c056-4228-8096-94e4461993e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:23.224462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156662d-c056-4228-8096-94e4461993e5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:23.264538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1188/2383] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:29:23.272473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fd46b-1d3b-48ae-9691-e76074339523 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:23.288878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fd46b-1d3b-48ae-9691-e76074339523 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:23.306457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccad7c3-3619-4984-a646-7166241d288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:23.307098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccad7c3-3619-4984-a646-7166241d288c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:23.307383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccad7c3-3619-4984-a646-7166241d288c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:23.315232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccad7c3-3619-4984-a646-7166241d288c admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:23.315905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccad7c3-3619-4984-a646-7166241d288c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:23.318558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccad7c3-3619-4984-a646-7166241d288c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:23.445166 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1196/2384] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:29:23 2022] POST /identity/v3/projects => generated 391 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:29:23.450916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156662d-c056-4228-8096-94e4461993e5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:23.469347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156662d-c056-4228-8096-94e4461993e5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:23.578202 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:23.631791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626d78-2371-4d1a-a39a-1018a1c7524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:23.641972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626d78-2371-4d1a-a39a-1018a1c7524c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:23.641972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626d78-2371-4d1a-a39a-1018a1c7524c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:23.641972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626d78-2371-4d1a-a39a-1018a1c7524c admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:23.641972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33626d78-2371-4d1a-a39a-1018a1c7524c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:23.641972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33626d78-2371-4d1a-a39a-1018a1c7524c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:23.930294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1189/2385] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:29:23 2022] POST /identity/v3/users => generated 296 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:29:23.934445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccad7c3-3619-4984-a646-7166241d288c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:23.948771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccad7c3-3619-4984-a646-7166241d288c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:23.954604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47f8c0-1c27-4df1-8f88-a3847e53d97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:23.955977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47f8c0-1c27-4df1-8f88-a3847e53d97c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:23.955977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47f8c0-1c27-4df1-8f88-a3847e53d97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:23.955977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47f8c0-1c27-4df1-8f88-a3847e53d97c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:23.956656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47f8c0-1c27-4df1-8f88-a3847e53d97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:23.958819 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47f8c0-1c27-4df1-8f88-a3847e53d97c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:23.972098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1197/2386] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:29:23.975848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626d78-2371-4d1a-a39a-1018a1c7524c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:23.992940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626d78-2371-4d1a-a39a-1018a1c7524c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:24.003746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8903b4-1a4a-4994-9d6c-b2133b49af09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:24.005355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8903b4-1a4a-4994-9d6c-b2133b49af09 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:24.005807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8903b4-1a4a-4994-9d6c-b2133b49af09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:24.008469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8903b4-1a4a-4994-9d6c-b2133b49af09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:24.009423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8903b4-1a4a-4994-9d6c-b2133b49af09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:24.021462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8903b4-1a4a-4994-9d6c-b2133b49af09 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:24.077949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1190/2387] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:29:24.081945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47f8c0-1c27-4df1-8f88-a3847e53d97c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:24.096722 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47f8c0-1c27-4df1-8f88-a3847e53d97c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:24.103594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d4390-a7f8-40fa-ab60-d04dda171feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:24.104686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d4390-a7f8-40fa-ab60-d04dda171feb admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:24.105130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d4390-a7f8-40fa-ab60-d04dda171feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:24.105620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d4390-a7f8-40fa-ab60-d04dda171feb admin admin] PATH_INFO: `/v3/projects/6cf3f2b6d31f4f47827aad18047c33fd/users/826d595c90bb424ab9a209f025c5e7db/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:24.128565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d4390-a7f8-40fa-ab60-d04dda171feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf3f2b6d31f4f47827aad18047c33fd, user_id=826d595c90bb424ab9a209f025c5e7db, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:24.131702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d4390-a7f8-40fa-ab60-d04dda171feb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:24.226778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1198/2388] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:29:24 2022] PUT /identity/v3/projects/6cf3f2b6d31f4f47827aad18047c33fd/users/826d595c90bb424ab9a209f025c5e7db/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:24.237003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd22d6-6f7c-44b3-8ac2-ba2cab8cf240 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:24.240048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd22d6-6f7c-44b3-8ac2-ba2cab8cf240 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:24.240048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd22d6-6f7c-44b3-8ac2-ba2cab8cf240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:24.282854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-15dd22d6-6f7c-44b3-8ac2-ba2cab8cf240 None None] MFA Rules not processed for user `826d595c90bb424ab9a209f025c5e7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:24.284295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-15dd22d6-6f7c-44b3-8ac2-ba2cab8cf240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:24.432545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1191/2389] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:24.458109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d4390-a7f8-40fa-ab60-d04dda171feb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:24.464311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4d4d4390-a7f8-40fa-ab60-d04dda171feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:24.595618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d4390-a7f8-40fa-ab60-d04dda171feb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:24.628149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f55350-2dc0-461a-af02-c7bdbbf51bc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:24.630234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f55350-2dc0-461a-af02-c7bdbbf51bc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:24.633571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f55350-2dc0-461a-af02-c7bdbbf51bc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:24.633571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f55350-2dc0-461a-af02-c7bdbbf51bc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:24.664045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f55350-2dc0-461a-af02-c7bdbbf51bc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:24.670849 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f55350-2dc0-461a-af02-c7bdbbf51bc4 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:24.701297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1199/2390] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:25.801995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd22d6-6f7c-44b3-8ac2-ba2cab8cf240 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:25.807566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-15dd22d6-6f7c-44b3-8ac2-ba2cab8cf240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:26.014488 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd22d6-6f7c-44b3-8ac2-ba2cab8cf240 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:26.019419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe012fe1-5525-43e0-9bba-af9dcf2af3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:26.020474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe012fe1-5525-43e0-9bba-af9dcf2af3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:26.021306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe012fe1-5525-43e0-9bba-af9dcf2af3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:26.022043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe012fe1-5525-43e0-9bba-af9dcf2af3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:26.026011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fe012fe1-5525-43e0-9bba-af9dcf2af3f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:26.124771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:26.146924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe012fe1-5525-43e0-9bba-af9dcf2af3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:26.148584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe012fe1-5525-43e0-9bba-af9dcf2af3f5 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:26.160348 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1192/2391] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:34.155099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674fd01-bd0b-4318-9812-f0e2b5b44e76 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:34.155605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674fd01-bd0b-4318-9812-f0e2b5b44e76 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:34.155699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674fd01-bd0b-4318-9812-f0e2b5b44e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:34.174997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9674fd01-bd0b-4318-9812-f0e2b5b44e76 None None] MFA Rules not processed for user `413dcd64b46f4e10947d9347dff434cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:34.176817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9674fd01-bd0b-4318-9812-f0e2b5b44e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:34.230817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1200/2392] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:36.449183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:37.787259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862b46-eb56-4598-84c8-82a89ab24a31 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:37.787903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862b46-eb56-4598-84c8-82a89ab24a31 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:37.787903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862b46-eb56-4598-84c8-82a89ab24a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:37.808150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ce862b46-eb56-4598-84c8-82a89ab24a31 None None] MFA Rules not processed for user `92712c36310c440c94da6c432c77a895`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:37.809351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ce862b46-eb56-4598-84c8-82a89ab24a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:37.867695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1193/2393] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:41.449721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674fd01-bd0b-4318-9812-f0e2b5b44e76 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:41.459594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674fd01-bd0b-4318-9812-f0e2b5b44e76 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:41.463222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c57972-8efc-466c-8bef-951a9c2ca442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:41.463817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c57972-8efc-466c-8bef-951a9c2ca442 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:41.464099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c57972-8efc-466c-8bef-951a9c2ca442 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:41.464372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c57972-8efc-466c-8bef-951a9c2ca442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:41.466210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a1c57972-8efc-466c-8bef-951a9c2ca442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:41.471139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:41.543228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c57972-8efc-466c-8bef-951a9c2ca442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:41.544707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c57972-8efc-466c-8bef-951a9c2ca442 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:41.556831 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1201/2394] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:42.671581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce862b46-eb56-4598-84c8-82a89ab24a31 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:42.673597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ce862b46-eb56-4598-84c8-82a89ab24a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:42.700870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce862b46-eb56-4598-84c8-82a89ab24a31 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:42.705044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:42.705511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:42.705768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:42.706093 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:42.708080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:42.733341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:42.735090 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:42.750568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1194/2395] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:29:47.618741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc963ae-024f-4a7c-ba9e-d3cbe4a66ed2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:47.619239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc963ae-024f-4a7c-ba9e-d3cbe4a66ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:47.619410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc963ae-024f-4a7c-ba9e-d3cbe4a66ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:47.642057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ecc963ae-024f-4a7c-ba9e-d3cbe4a66ed2 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:47.643529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ecc963ae-024f-4a7c-ba9e-d3cbe4a66ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:47.734332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1202/2396] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:29:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:47.748636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:47.750175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:47.826764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7716138-cd27-4805-b36b-f88cb6814080 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:47.832716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac18c31-5319-4fd3-bba7-f4b1817b3365 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:47.833271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac18c31-5319-4fd3-bba7-f4b1817b3365 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:47.833549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac18c31-5319-4fd3-bba7-f4b1817b3365 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:47.833907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac18c31-5319-4fd3-bba7-f4b1817b3365 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:47.842526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac18c31-5319-4fd3-bba7-f4b1817b3365 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:47.844143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac18c31-5319-4fd3-bba7-f4b1817b3365 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:47.852810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1195/2397] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:29:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 11:29:51.829634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc963ae-024f-4a7c-ba9e-d3cbe4a66ed2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:51.831144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ecc963ae-024f-4a7c-ba9e-d3cbe4a66ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:51.847851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc963ae-024f-4a7c-ba9e-d3cbe4a66ed2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:51.853461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e1734-a967-47ab-8646-e27628fdbe2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:51.854099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e1734-a967-47ab-8646-e27628fdbe2a service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:51.854377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e1734-a967-47ab-8646-e27628fdbe2a service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:51.854650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e1734-a967-47ab-8646-e27628fdbe2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:51.856330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-560e1734-a967-47ab-8646-e27628fdbe2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:51.927265 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e1734-a967-47ab-8646-e27628fdbe2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:51.928892 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e1734-a967-47ab-8646-e27628fdbe2a service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:51.939538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1203/2398] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:53.211018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac18c31-5319-4fd3-bba7-f4b1817b3365 service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:53.223494 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac18c31-5319-4fd3-bba7-f4b1817b3365 service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:53.235387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83055758-ec38-4279-9df4-a89694ce4d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:53.236550 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83055758-ec38-4279-9df4-a89694ce4d06 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:53.237133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83055758-ec38-4279-9df4-a89694ce4d06 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:53.238373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83055758-ec38-4279-9df4-a89694ce4d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:53.242312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-83055758-ec38-4279-9df4-a89694ce4d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:53.257134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83055758-ec38-4279-9df4-a89694ce4d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:53.259665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83055758-ec38-4279-9df4-a89694ce4d06 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:53.272740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1196/2399] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:55.010245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e1734-a967-47ab-8646-e27628fdbe2a service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:55.055022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:55.079818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e1734-a967-47ab-8646-e27628fdbe2a service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:55.090910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:55.090910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:55.090910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:55.090910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] PATH_INFO: `/v3/users/826d595c90bb424ab9a209f025c5e7db` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:55.091325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=826d595c90bb424ab9a209f025c5e7db)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:55.091325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:55.313753 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '826d595c90bb424ab9a209f025c5e7db'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:55.314520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '826d595c90bb424ab9a209f025c5e7db'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:55.315135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '826d595c90bb424ab9a209f025c5e7db'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:55.335617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '826d595c90bb424ab9a209f025c5e7db'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:55.350247 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '826d595c90bb424ab9a209f025c5e7db'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:55.424213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1204/2400] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:29:55 2022] DELETE /identity/v3/users/826d595c90bb424ab9a209f025c5e7db => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:29:55.429726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8bba5-1bc9-4fdd-8016-b91db782cada None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:55.430609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8bba5-1bc9-4fdd-8016-b91db782cada None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:55.430782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8bba5-1bc9-4fdd-8016-b91db782cada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:55.450483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-47b8bba5-1bc9-4fdd-8016-b91db782cada None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:55.451692 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-47b8bba5-1bc9-4fdd-8016-b91db782cada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:55.556469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1197/2401] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:55.566858 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:55.568222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:55.656418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc29a0c-0fe3-49f2-9eaa-c4eb9a7893a6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:55.661099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05340e8-1744-4168-b9d6-07875e9600d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:55.662102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05340e8-1744-4168-b9d6-07875e9600d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:55.662596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05340e8-1744-4168-b9d6-07875e9600d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:55.663067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05340e8-1744-4168-b9d6-07875e9600d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:55.673410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05340e8-1744-4168-b9d6-07875e9600d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:55.676061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05340e8-1744-4168-b9d6-07875e9600d1 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:55.689422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1205/2402] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:56.270270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8bba5-1bc9-4fdd-8016-b91db782cada None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:56.272435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-47b8bba5-1bc9-4fdd-8016-b91db782cada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:56.289825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8bba5-1bc9-4fdd-8016-b91db782cada None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:56.304908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:56.304908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:56.304908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:56.304908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] PATH_INFO: `/v3/projects/6cf3f2b6d31f4f47827aad18047c33fd` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:56.305281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cf3f2b6d31f4f47827aad18047c33fd)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:56.307862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:56.319153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:56.485785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cf3f2b6d31f4f47827aad18047c33fd', 'request_id': 'req-045454c3-d161-4077-80bb-969c51845af4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:56.493336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cf3f2b6d31f4f47827aad18047c33fd', 'request_id': 'req-045454c3-d161-4077-80bb-969c51845af4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:56.500456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cf3f2b6d31f4f47827aad18047c33fd', 'request_id': 'req-045454c3-d161-4077-80bb-969c51845af4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:56.595287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] The token cache is being invalidate because project 6cf3f2b6d31f4f47827aad18047c33fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:29:56.595585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:56.596222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:29:56.597728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1198/2403] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:29:56 2022] DELETE /identity/v3/projects/6cf3f2b6d31f4f47827aad18047c33fd => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:30:04.794874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05340e8-1744-4168-b9d6-07875e9600d1 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:30:04.796833 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d05340e8-1744-4168-b9d6-07875e9600d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:04.801356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:30:04.804018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:04.809014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05340e8-1744-4168-b9d6-07875e9600d1 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:30:04.864103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868ffcf-f0b2-42fd-aff9-a9e4da41d461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:30:04.865224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868ffcf-f0b2-42fd-aff9-a9e4da41d461 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:30:04.865506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868ffcf-f0b2-42fd-aff9-a9e4da41d461 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:30:04.865780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868ffcf-f0b2-42fd-aff9-a9e4da41d461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:30:04.867586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7868ffcf-f0b2-42fd-aff9-a9e4da41d461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:04.867868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045454c3-d161-4077-80bb-969c51845af4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:30:04.871598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2b29c-8547-49d0-8943-5262240e5403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:30:04.872201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2b29c-8547-49d0-8943-5262240e5403 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:30:04.872479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2b29c-8547-49d0-8943-5262240e5403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:30:04.872746 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2b29c-8547-49d0-8943-5262240e5403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:30:04.877545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868ffcf-f0b2-42fd-aff9-a9e4da41d461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:30:04.879031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868ffcf-f0b2-42fd-aff9-a9e4da41d461 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:30:04.880519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2b29c-8547-49d0-8943-5262240e5403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:30:04.881984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2b29c-8547-49d0-8943-5262240e5403 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:30:04.884187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:30:04.888138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1206/2404] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:30:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:30:04.889016 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868ffcf-f0b2-42fd-aff9-a9e4da41d461 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:30:04.892512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1199/2405] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:30:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:30:04.899408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2b29c-8547-49d0-8943-5262240e5403 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:30:04.899408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-35f2b29c-8547-49d0-8943-5262240e5403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:04.899784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868ffcf-f0b2-42fd-aff9-a9e4da41d461 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:30:04.902470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19daec83-9faa-4a1a-af15-8b01c1b0128b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:30:04.903154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daec83-9faa-4a1a-af15-8b01c1b0128b service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:30:04.903510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daec83-9faa-4a1a-af15-8b01c1b0128b service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:30:04.903979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daec83-9faa-4a1a-af15-8b01c1b0128b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:30:04.911326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2b29c-8547-49d0-8943-5262240e5403 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:30:04.912601 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19daec83-9faa-4a1a-af15-8b01c1b0128b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:30:04.914165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19daec83-9faa-4a1a-af15-8b01c1b0128b service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:30:04.915653 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a4be1-a0f7-4521-babe-59c616e19a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:30:04.916460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a4be1-a0f7-4521-babe-59c616e19a9e service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:30:04.916588 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a4be1-a0f7-4521-babe-59c616e19a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:30:04.916904 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a4be1-a0f7-4521-babe-59c616e19a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:30:04.923816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1207/2406] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:30:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:30:04.926505 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a4be1-a0f7-4521-babe-59c616e19a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:30:04.928433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a4be1-a0f7-4521-babe-59c616e19a9e service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:30:04.936991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1200/2407] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:30:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:30:14.933170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:30:14.977765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:30:24.881418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19daec83-9faa-4a1a-af15-8b01c1b0128b service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:30:24.883480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-19daec83-9faa-4a1a-af15-8b01c1b0128b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:24.972310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19daec83-9faa-4a1a-af15-8b01c1b0128b service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:30:24.977939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd3d1-de50-438a-b8c9-7cbc50c3dbbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:30:24.978617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd3d1-de50-438a-b8c9-7cbc50c3dbbc service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:30:24.978960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd3d1-de50-438a-b8c9-7cbc50c3dbbc service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:30:24.979293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd3d1-de50-438a-b8c9-7cbc50c3dbbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:30:24.981073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-33cbd3d1-de50-438a-b8c9-7cbc50c3dbbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:24.993475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbd3d1-de50-438a-b8c9-7cbc50c3dbbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:30:24.995123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbd3d1-de50-438a-b8c9-7cbc50c3dbbc service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:30:25.006777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1208/2408] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:30:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:30:34.499547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a4be1-a0f7-4521-babe-59c616e19a9e service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:30:34.508574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a4be1-a0f7-4521-babe-59c616e19a9e service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:30:34.512338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e031a74-9f17-460b-8b28-3cd60f599b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:30:34.512933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e031a74-9f17-460b-8b28-3cd60f599b51 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:30:34.513218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e031a74-9f17-460b-8b28-3cd60f599b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:30:34.513493 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e031a74-9f17-460b-8b28-3cd60f599b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:30:34.515300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0e031a74-9f17-460b-8b28-3cd60f599b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:34.584691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e031a74-9f17-460b-8b28-3cd60f599b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:30:34.586419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e031a74-9f17-460b-8b28-3cd60f599b51 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:30:34.596618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1201/2409] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:30:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:30:35.001736 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=34 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:30:40.579688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd3d1-de50-438a-b8c9-7cbc50c3dbbc service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:30:40.604857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd3d1-de50-438a-b8c9-7cbc50c3dbbc service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:30:40.611028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d2785-1e2f-4858-a62e-a0ae6c9e38a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:30:40.612900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d2785-1e2f-4858-a62e-a0ae6c9e38a1 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:30:40.612900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d2785-1e2f-4858-a62e-a0ae6c9e38a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:30:40.612900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d2785-1e2f-4858-a62e-a0ae6c9e38a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:30:40.616087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-277d2785-1e2f-4858-a62e-a0ae6c9e38a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:30:40.631525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d2785-1e2f-4858-a62e-a0ae6c9e38a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:30:40.633368 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d2785-1e2f-4858-a62e-a0ae6c9e38a1 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:30:40.664544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1209/2410] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:30:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:30:44.592106 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=30 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:30:50.641611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:53.693165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7504a-5325-4de1-843d-fd7b075a69e2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:53.693702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7504a-5325-4de1-843d-fd7b075a69e2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:53.693702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7504a-5325-4de1-843d-fd7b075a69e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:53.713038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-42c7504a-5325-4de1-843d-fd7b075a69e2 None None] MFA Rules not processed for user `57be68e76fc14d2ca309571e284509bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:53.714304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-42c7504a-5325-4de1-843d-fd7b075a69e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:53.768987 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1202/2411] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:31:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:57.680756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d2785-1e2f-4858-a62e-a0ae6c9e38a1 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:57.692345 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d2785-1e2f-4858-a62e-a0ae6c9e38a1 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:57.698406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047f58b-ef53-4262-bc9c-d3b2e84a521b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:57.699060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047f58b-ef53-4262-bc9c-d3b2e84a521b service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:57.699338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047f58b-ef53-4262-bc9c-d3b2e84a521b service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:57.699618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047f58b-ef53-4262-bc9c-d3b2e84a521b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:57.701445 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2047f58b-ef53-4262-bc9c-d3b2e84a521b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:57.714017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047f58b-ef53-4262-bc9c-d3b2e84a521b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:57.715650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047f58b-ef53-4262-bc9c-d3b2e84a521b service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:57.725780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1210/2412] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:03.761610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=25 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:07.721438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:28.312905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49fb89-6014-4ce9-b4f2-c71fdcce1a00 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:28.313439 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49fb89-6014-4ce9-b4f2-c71fdcce1a00 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:28.313439 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49fb89-6014-4ce9-b4f2-c71fdcce1a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:28.333330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cd49fb89-6014-4ce9-b4f2-c71fdcce1a00 None None] MFA Rules not processed for user `92712c36310c440c94da6c432c77a895`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:28.334624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cd49fb89-6014-4ce9-b4f2-c71fdcce1a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:28.390260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1203/2413] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:32:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:28.695981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047f58b-ef53-4262-bc9c-d3b2e84a521b service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:28.706071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047f58b-ef53-4262-bc9c-d3b2e84a521b service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:28.709684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:28.710353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:28.710634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:28.710930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:28.712676 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:28.722451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:28.723876 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:28.731769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1211/2414] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:29.150262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49fb89-6014-4ce9-b4f2-c71fdcce1a00 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:29.151872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cd49fb89-6014-4ce9-b4f2-c71fdcce1a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:29.232057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49fb89-6014-4ce9-b4f2-c71fdcce1a00 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:29.237685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a5ad6-b2b2-49aa-babe-937cdc313c26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:29.238329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a5ad6-b2b2-49aa-babe-937cdc313c26 service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:29.238611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a5ad6-b2b2-49aa-babe-937cdc313c26 service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:29.238884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a5ad6-b2b2-49aa-babe-937cdc313c26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:29.240635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ce3a5ad6-b2b2-49aa-babe-937cdc313c26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:29.253072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a5ad6-b2b2-49aa-babe-937cdc313c26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:29.254736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a5ad6-b2b2-49aa-babe-937cdc313c26 service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:29.266158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1204/2415] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:38.728328 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:39.261341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=57 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:39.768290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:39.770232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:39.789918 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c646a-60be-4745-ba31-8c4a6e416c5f service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:39.801645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078eb21-793a-4bdf-b5fa-f7e704b10432 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:39.803902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078eb21-793a-4bdf-b5fa-f7e704b10432 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:39.804668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078eb21-793a-4bdf-b5fa-f7e704b10432 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:39.805072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078eb21-793a-4bdf-b5fa-f7e704b10432 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:39.807264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d078eb21-793a-4bdf-b5fa-f7e704b10432 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:39.821947 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078eb21-793a-4bdf-b5fa-f7e704b10432 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:39.823696 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078eb21-793a-4bdf-b5fa-f7e704b10432 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:39.837079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1212/2416] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:39.927080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a5ad6-b2b2-49aa-babe-937cdc313c26 service placement] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:39.936912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a5ad6-b2b2-49aa-babe-937cdc313c26 service placement] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:39.940752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fe5d3-103e-44f3-9aa7-84f7e55a45d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:39.941419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fe5d3-103e-44f3-9aa7-84f7e55a45d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:39.941820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fe5d3-103e-44f3-9aa7-84f7e55a45d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:39.942184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fe5d3-103e-44f3-9aa7-84f7e55a45d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:39.944092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-265fe5d3-103e-44f3-9aa7-84f7e55a45d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:39.955624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fe5d3-103e-44f3-9aa7-84f7e55a45d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:39.960925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fe5d3-103e-44f3-9aa7-84f7e55a45d5 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:39.966335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1205/2417] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:40.387742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9ac1e-58da-493b-8dd4-664f83cb9127 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:40.388048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9ac1e-58da-493b-8dd4-664f83cb9127 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:40.388362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9ac1e-58da-493b-8dd4-664f83cb9127 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:40.389796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1213/2418] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:32:40.396634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48291b-5751-463b-8167-a7d4d6e918ca None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:40.398026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48291b-5751-463b-8167-a7d4d6e918ca None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:40.398026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48291b-5751-463b-8167-a7d4d6e918ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:40.438708 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cd48291b-5751-463b-8167-a7d4d6e918ca None None] MFA Rules not processed for user `b0145242ef784168bada8348ccd95f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:40.439928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cd48291b-5751-463b-8167-a7d4d6e918ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:40.450171 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1206/2419] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:32:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:40.462610 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9ac1e-58da-493b-8dd4-664f83cb9127 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:40.474702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9ac1e-58da-493b-8dd4-664f83cb9127 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:40.478396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03924d39-7b4e-48b9-8fc6-c0691a5f2622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:40.479005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03924d39-7b4e-48b9-8fc6-c0691a5f2622 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:40.479279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03924d39-7b4e-48b9-8fc6-c0691a5f2622 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:40.479687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03924d39-7b4e-48b9-8fc6-c0691a5f2622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:40.488022 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03924d39-7b4e-48b9-8fc6-c0691a5f2622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:40.491145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03924d39-7b4e-48b9-8fc6-c0691a5f2622 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:40.497609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1214/2420] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:40.585910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2c1db-b90d-4e2d-ae2f-8379bdfdaf0d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:40.586189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2c1db-b90d-4e2d-ae2f-8379bdfdaf0d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:40.586474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2c1db-b90d-4e2d-ae2f-8379bdfdaf0d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:40.587896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1207/2421] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:32:40.593433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe87e85-4549-425a-957c-ec2e40faebe0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:40.594544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe87e85-4549-425a-957c-ec2e40faebe0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:40.594544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe87e85-4549-425a-957c-ec2e40faebe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:40.627855 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1fe87e85-4549-425a-957c-ec2e40faebe0 None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:40.629216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1fe87e85-4549-425a-957c-ec2e40faebe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:40.693243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1215/2422] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:32:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:40.703837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2c1db-b90d-4e2d-ae2f-8379bdfdaf0d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:40.714973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2c1db-b90d-4e2d-ae2f-8379bdfdaf0d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:40.719070 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983bccd-85d8-43f8-b871-05014a60ddbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:40.719661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983bccd-85d8-43f8-b871-05014a60ddbd service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:40.719940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983bccd-85d8-43f8-b871-05014a60ddbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:40.720214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983bccd-85d8-43f8-b871-05014a60ddbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:40.730672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983bccd-85d8-43f8-b871-05014a60ddbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:40.732154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983bccd-85d8-43f8-b871-05014a60ddbd service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:40.743513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1208/2423] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:47.553920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe87e85-4549-425a-957c-ec2e40faebe0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:47.565374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe87e85-4549-425a-957c-ec2e40faebe0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:47.571366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:47.571975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:47.572259 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:47.572535 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:47.574338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:47.641974 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:47.643649 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:47.656966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1216/2424] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:50.738947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=25 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:57.649908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=71 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:08.683340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078956f-5616-471f-ac1b-f59f206bfdf8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:08.683905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078956f-5616-471f-ac1b-f59f206bfdf8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:08.683905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078956f-5616-471f-ac1b-f59f206bfdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:08.703825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5078956f-5616-471f-ac1b-f59f206bfdf8 None None] MFA Rules not processed for user `cd79bc9c3a694fe2a4e3c645e682e438`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:08.705027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5078956f-5616-471f-ac1b-f59f206bfdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:08.762075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1209/2425] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:33:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:18.754849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=25 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:40.965340 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:40.971515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:41.018673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29821840-b0c7-4b4e-ae07-0edc325f1bf9 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:41.024377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:41.025082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:41.025398 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:41.025675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:41.028064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:41.042788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:41.044732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:41.056381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1217/2426] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:42.469279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078956f-5616-471f-ac1b-f59f206bfdf8 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:42.491399 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078956f-5616-471f-ac1b-f59f206bfdf8 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:42.495257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a31cbe-3d7c-460c-822e-339125f52462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:42.495884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a31cbe-3d7c-460c-822e-339125f52462 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:42.496165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a31cbe-3d7c-460c-822e-339125f52462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:42.496434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a31cbe-3d7c-460c-822e-339125f52462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:42.498274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-47a31cbe-3d7c-460c-822e-339125f52462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:42.590943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a31cbe-3d7c-460c-822e-339125f52462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:42.592383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a31cbe-3d7c-460c-822e-339125f52462 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:42.612893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1210/2427] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:46.515250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:46.517082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:46.531875 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faa9ec-0fb2-4725-aa13-880115ea2a07 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:46.540214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:46.540214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:46.540214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:46.540214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] PATH_INFO: `/v3/users/57be68e76fc14d2ca309571e284509bd` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:46.540611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57be68e76fc14d2ca309571e284509bd)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:46.540809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:46.819400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57be68e76fc14d2ca309571e284509bd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.821227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57be68e76fc14d2ca309571e284509bd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.821227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57be68e76fc14d2ca309571e284509bd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.828277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57be68e76fc14d2ca309571e284509bd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.840077 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-63f97d3f-2ee8-4dea-8f8c-15f947ae1fe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57be68e76fc14d2ca309571e284509bd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.948918 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1218/2428] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:46 2022] DELETE /identity/v3/users/57be68e76fc14d2ca309571e284509bd => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:47.177629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a31cbe-3d7c-460c-822e-339125f52462 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:47.190722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a31cbe-3d7c-460c-822e-339125f52462 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:47.251832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:47.252552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.253017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.253293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] PATH_INFO: `/v3/projects/e2a13f3ca60643a9a3507476325d351a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.254464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2a13f3ca60643a9a3507476325d351a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:47.256403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:47.411467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2a13f3ca60643a9a3507476325d351a', 'request_id': 'req-e02863e4-a551-4726-84e2-de8e39cd7616'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:47.418470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2a13f3ca60643a9a3507476325d351a', 'request_id': 'req-e02863e4-a551-4726-84e2-de8e39cd7616'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:47.419694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:47.426575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2a13f3ca60643a9a3507476325d351a', 'request_id': 'req-e02863e4-a551-4726-84e2-de8e39cd7616'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:47.503699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] The token cache is being invalidate because project e2a13f3ca60643a9a3507476325d351a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:33:47.503904 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:47.504621 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e02863e4-a551-4726-84e2-de8e39cd7616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:47.510239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1211/2429] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:33:47 2022] DELETE /identity/v3/projects/e2a13f3ca60643a9a3507476325d351a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:47.512660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e8794-34a9-4cbe-979a-7456051b8cd2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.513158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e8794-34a9-4cbe-979a-7456051b8cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.513581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e8794-34a9-4cbe-979a-7456051b8cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.568051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-848e8794-34a9-4cbe-979a-7456051b8cd2 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:47.569613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-848e8794-34a9-4cbe-979a-7456051b8cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:47.631172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1219/2430] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:47.639676 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944e0dc-d069-48ea-8759-6952000a3e1f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.639955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944e0dc-d069-48ea-8759-6952000a3e1f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.640225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944e0dc-d069-48ea-8759-6952000a3e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.659858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2944e0dc-d069-48ea-8759-6952000a3e1f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:47.661123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2944e0dc-d069-48ea-8759-6952000a3e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:47.668857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1212/2431] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:47.676932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e8794-34a9-4cbe-979a-7456051b8cd2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:47.687580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e8794-34a9-4cbe-979a-7456051b8cd2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:47.692518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941e6f2-1b13-4fce-b367-718bbfff7e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:47.693294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941e6f2-1b13-4fce-b367-718bbfff7e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.693584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941e6f2-1b13-4fce-b367-718bbfff7e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.693861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941e6f2-1b13-4fce-b367-718bbfff7e72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.694548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941e6f2-1b13-4fce-b367-718bbfff7e72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:33:47.694855 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941e6f2-1b13-4fce-b367-718bbfff7e72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:47.696469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941e6f2-1b13-4fce-b367-718bbfff7e72 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:47.710302 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1220/2432] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:33:47.717178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944e0dc-d069-48ea-8759-6952000a3e1f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:47.726640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944e0dc-d069-48ea-8759-6952000a3e1f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:47.731396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b607f49-502d-464b-8966-1519def8f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:47.731993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b607f49-502d-464b-8966-1519def8f709 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.732271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b607f49-502d-464b-8966-1519def8f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.732536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b607f49-502d-464b-8966-1519def8f709 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.733158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b607f49-502d-464b-8966-1519def8f709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:47.735099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b607f49-502d-464b-8966-1519def8f709 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:47.852190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1213/2433] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:47 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:47.857273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941e6f2-1b13-4fce-b367-718bbfff7e72 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:47.864040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:47.868609 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941e6f2-1b13-4fce-b367-718bbfff7e72 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:47.896142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838c4d8-7cc0-473d-bb1d-a1580977074d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.896425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838c4d8-7cc0-473d-bb1d-a1580977074d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.896696 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838c4d8-7cc0-473d-bb1d-a1580977074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.927311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a838c4d8-7cc0-473d-bb1d-a1580977074d None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:47.928519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a838c4d8-7cc0-473d-bb1d-a1580977074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:47.930116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8265b-ffcc-440c-a65d-247864083399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:47.930759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8265b-ffcc-440c-a65d-247864083399 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.931011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8265b-ffcc-440c-a65d-247864083399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.931285 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8265b-ffcc-440c-a65d-247864083399 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.931926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8265b-ffcc-440c-a65d-247864083399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:47.934509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8265b-ffcc-440c-a65d-247864083399 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:47.984753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1214/2434] 10.222.0.83 () {58 vars in 967 bytes} [Wed Aug 24 11:33:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:48.028888 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838c4d8-7cc0-473d-bb1d-a1580977074d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.030329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a838c4d8-7cc0-473d-bb1d-a1580977074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:48.081907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1221/2435] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:47 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:48.083397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8265b-ffcc-440c-a65d-247864083399 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.093386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8265b-ffcc-440c-a65d-247864083399 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.098459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21298da8-4aa7-4a5f-a07f-fbb0117041a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.099065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21298da8-4aa7-4a5f-a07f-fbb0117041a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.099346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21298da8-4aa7-4a5f-a07f-fbb0117041a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.099626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21298da8-4aa7-4a5f-a07f-fbb0117041a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.100378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21298da8-4aa7-4a5f-a07f-fbb0117041a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.102386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21298da8-4aa7-4a5f-a07f-fbb0117041a8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.118476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1222/2436] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:33:48.119405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21298da8-4aa7-4a5f-a07f-fbb0117041a8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.119585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838c4d8-7cc0-473d-bb1d-a1580977074d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.125391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3fcd8-8ade-46bf-906a-ec26f167bc8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.126039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3fcd8-8ade-46bf-906a-ec26f167bc8e service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.126334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3fcd8-8ade-46bf-906a-ec26f167bc8e service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.126602 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3fcd8-8ade-46bf-906a-ec26f167bc8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.130437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21298da8-4aa7-4a5f-a07f-fbb0117041a8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.135265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3fcd8-8ade-46bf-906a-ec26f167bc8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.135397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be33b-0f24-4902-9fa6-820021e3402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.135928 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be33b-0f24-4902-9fa6-820021e3402d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.136213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be33b-0f24-4902-9fa6-820021e3402d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.136487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be33b-0f24-4902-9fa6-820021e3402d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.136881 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3fcd8-8ade-46bf-906a-ec26f167bc8e service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.137164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4be33b-0f24-4902-9fa6-820021e3402d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.138812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4be33b-0f24-4902-9fa6-820021e3402d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.145363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1215/2437] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:33:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 11:33:48.152571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1223/2438] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:33:48.153620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3fcd8-8ade-46bf-906a-ec26f167bc8e service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.164333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3fcd8-8ade-46bf-906a-ec26f167bc8e service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.169240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2970227-e8a4-4b9c-bebe-9796b8616c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.169919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2970227-e8a4-4b9c-bebe-9796b8616c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.174015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2970227-e8a4-4b9c-bebe-9796b8616c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.174015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2970227-e8a4-4b9c-bebe-9796b8616c91 admin admin] PATH_INFO: `/v3/projects/de6ccf52d6394c01859a5ccf0774f2d8/users/4dfbe3078d714129b5a6e6e45dbc0b25/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.189055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2970227-e8a4-4b9c-bebe-9796b8616c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6ccf52d6394c01859a5ccf0774f2d8, user_id=4dfbe3078d714129b5a6e6e45dbc0b25, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.193020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2970227-e8a4-4b9c-bebe-9796b8616c91 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.279504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1216/2439] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:48 2022] PUT /identity/v3/projects/de6ccf52d6394c01859a5ccf0774f2d8/users/4dfbe3078d714129b5a6e6e45dbc0b25/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:33:48.290545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54150e77-9262-4b88-b9b1-f01ed7c125d3 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.290834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54150e77-9262-4b88-b9b1-f01ed7c125d3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.291108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54150e77-9262-4b88-b9b1-f01ed7c125d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.314792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-54150e77-9262-4b88-b9b1-f01ed7c125d3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:48.316010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-54150e77-9262-4b88-b9b1-f01ed7c125d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:48.390611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1224/2440] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:48.400581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2970227-e8a4-4b9c-bebe-9796b8616c91 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.402498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a2970227-e8a4-4b9c-bebe-9796b8616c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:48.429098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:48.482741 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2970227-e8a4-4b9c-bebe-9796b8616c91 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.486396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f491a6f-2acb-41fa-899f-c5c9dfb09c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.486989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f491a6f-2acb-41fa-899f-c5c9dfb09c59 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.487265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f491a6f-2acb-41fa-899f-c5c9dfb09c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.487536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f491a6f-2acb-41fa-899f-c5c9dfb09c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.497386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f491a6f-2acb-41fa-899f-c5c9dfb09c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.498893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f491a6f-2acb-41fa-899f-c5c9dfb09c59 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.508827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1217/2441] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:53.983062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54150e77-9262-4b88-b9b1-f01ed7c125d3 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:53.985739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-54150e77-9262-4b88-b9b1-f01ed7c125d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:54.085682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54150e77-9262-4b88-b9b1-f01ed7c125d3 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:54.093246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:54.094019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:54.094378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:54.094716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:54.096954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:54.112128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:54.114486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:54.126220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1225/2442] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:33:55.322288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dba59-0cf8-48e7-a3fa-628555434922 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:55.323243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dba59-0cf8-48e7-a3fa-628555434922 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:55.323369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dba59-0cf8-48e7-a3fa-628555434922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:55.359492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-855dba59-0cf8-48e7-a3fa-628555434922 None None] MFA Rules not processed for user `4dfbe3078d714129b5a6e6e45dbc0b25`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:55.360859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-855dba59-0cf8-48e7-a3fa-628555434922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:55.455941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1218/2443] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:33:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:55.474749 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:55.476730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:55.483956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:55.578294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebf8ca-5055-45a2-a397-8e669e0c69f8 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:55.584403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3871a-1d16-42bf-b4f4-7f00e523f248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:55.585039 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3871a-1d16-42bf-b4f4-7f00e523f248 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:55.585319 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3871a-1d16-42bf-b4f4-7f00e523f248 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:55.586056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3871a-1d16-42bf-b4f4-7f00e523f248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:55.604672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3871a-1d16-42bf-b4f4-7f00e523f248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:55.606353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3871a-1d16-42bf-b4f4-7f00e523f248 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:55.616765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1226/2444] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:58.747541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09b931-5f2d-4e02-b813-086d5997c232 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:58.748036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09b931-5f2d-4e02-b813-086d5997c232 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:58.748295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09b931-5f2d-4e02-b813-086d5997c232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:58.779928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2c09b931-5f2d-4e02-b813-086d5997c232 None None] MFA Rules not processed for user `4dfbe3078d714129b5a6e6e45dbc0b25`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:58.780197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3871a-1d16-42bf-b4f4-7f00e523f248 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:58.781231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2c09b931-5f2d-4e02-b813-086d5997c232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:58.782548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-33f3871a-1d16-42bf-b4f4-7f00e523f248 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:58.790840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1219/2445] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:33:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:58.811130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3871a-1d16-42bf-b4f4-7f00e523f248 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:58.816569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:58.817344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:58.817892 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:58.818286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] PATH_INFO: `/v3/users/413dcd64b46f4e10947d9347dff434cc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:58.819772 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=413dcd64b46f4e10947d9347dff434cc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:58.822472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:59.088275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413dcd64b46f4e10947d9347dff434cc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.090332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413dcd64b46f4e10947d9347dff434cc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.091006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413dcd64b46f4e10947d9347dff434cc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.098630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413dcd64b46f4e10947d9347dff434cc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.111815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413dcd64b46f4e10947d9347dff434cc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.176503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1227/2446] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:58 2022] DELETE /identity/v3/users/413dcd64b46f4e10947d9347dff434cc => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:59.406647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09b931-5f2d-4e02-b813-086d5997c232 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:59.430180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09b931-5f2d-4e02-b813-086d5997c232 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:59.502677 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:59.503435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:59.503788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:59.504133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] PATH_INFO: `/v3/projects/7e02d1127ce24c20822ab10c78edb00b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:59.505452 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e02d1127ce24c20822ab10c78edb00b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:59.507832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:59.656773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:59.666668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e02d1127ce24c20822ab10c78edb00b', 'request_id': 'req-89164438-e8b9-40d5-a510-c879d6be12b5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.675468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e02d1127ce24c20822ab10c78edb00b', 'request_id': 'req-89164438-e8b9-40d5-a510-c879d6be12b5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.683893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e02d1127ce24c20822ab10c78edb00b', 'request_id': 'req-89164438-e8b9-40d5-a510-c879d6be12b5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.756903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] The token cache is being invalidate because project 7e02d1127ce24c20822ab10c78edb00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:33:59.757617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.758604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:59.761810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1220/2447] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:33:59 2022] DELETE /identity/v3/projects/7e02d1127ce24c20822ab10c78edb00b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:59.766640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:59.772124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:59.784204 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc1b32-c131-457a-ad3b-cca407e938b5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:59.845624 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:59.846223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:59.846508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:59.846785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] PATH_INFO: `/v3/users/1518cd7f166b474d8d8f6557c02d4f75` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:59.847930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1518cd7f166b474d8d8f6557c02d4f75)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:59.849933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:59.851457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:00.086977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1518cd7f166b474d8d8f6557c02d4f75'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.087637 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1518cd7f166b474d8d8f6557c02d4f75'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.088250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1518cd7f166b474d8d8f6557c02d4f75'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.095107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1518cd7f166b474d8d8f6557c02d4f75'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.108360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5efaa754-5657-4fdf-bebc-6e4780315410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1518cd7f166b474d8d8f6557c02d4f75'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.210134 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1228/2448] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:59 2022] DELETE /identity/v3/users/1518cd7f166b474d8d8f6557c02d4f75 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:34:00.474821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:00.476383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:00.491035 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89164438-e8b9-40d5-a510-c879d6be12b5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:00.550002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:00.550637 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:00.550919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:00.551200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] PATH_INFO: `/v3/projects/12ee7a05cead40078bd518b096cc7023` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:00.552355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ee7a05cead40078bd518b096cc7023)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:00.554318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:00.684757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ee7a05cead40078bd518b096cc7023', 'request_id': 'req-b5442154-6ac3-487c-85ed-0f2e7fafe00f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.692264 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ee7a05cead40078bd518b096cc7023', 'request_id': 'req-b5442154-6ac3-487c-85ed-0f2e7fafe00f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.698626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ee7a05cead40078bd518b096cc7023', 'request_id': 'req-b5442154-6ac3-487c-85ed-0f2e7fafe00f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.798976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] The token cache is being invalidate because project 12ee7a05cead40078bd518b096cc7023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:34:00.799646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.799967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:00.801514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1221/2449] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:34:00 2022] DELETE /identity/v3/projects/12ee7a05cead40078bd518b096cc7023 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:10.206482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:10.795622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:40.360709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ced7b-7aa8-4691-a4b6-b97d5b2cc4e2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:40.361241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ced7b-7aa8-4691-a4b6-b97d5b2cc4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:40.361351 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ced7b-7aa8-4691-a4b6-b97d5b2cc4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:40.402446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fb1ced7b-7aa8-4691-a4b6-b97d5b2cc4e2 None None] MFA Rules not processed for user `92712c36310c440c94da6c432c77a895`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:40.403679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fb1ced7b-7aa8-4691-a4b6-b97d5b2cc4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:40.460011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1229/2450] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:34:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:34:41.234449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:41.235898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:41.301675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5442154-6ac3-487c-85ed-0f2e7fafe00f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:41.305594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:41.306226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:41.306510 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:41.306804 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:41.308512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:41.380415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:41.381843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:41.393470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1222/2451] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:42.505998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ced7b-7aa8-4691-a4b6-b97d5b2cc4e2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:42.507457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fb1ced7b-7aa8-4691-a4b6-b97d5b2cc4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:42.520965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ced7b-7aa8-4691-a4b6-b97d5b2cc4e2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:42.525375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c177a-16c3-4c6f-a96e-502e3b991b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:42.526013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c177a-16c3-4c6f-a96e-502e3b991b09 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:42.526294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c177a-16c3-4c6f-a96e-502e3b991b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:42.526565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c177a-16c3-4c6f-a96e-502e3b991b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:42.528329 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-819c177a-16c3-4c6f-a96e-502e3b991b09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:42.543797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c177a-16c3-4c6f-a96e-502e3b991b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:42.548912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c177a-16c3-4c6f-a96e-502e3b991b09 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:42.557401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1230/2452] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:51.388345 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=54 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:52.552118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=40 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:53.122875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:53.124389 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:53.138370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e841e-1c90-4fbc-90f1-b39a058a989f service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:53.144152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b7bec-4fd2-4f12-87e7-ac543555ed6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:53.144738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b7bec-4fd2-4f12-87e7-ac543555ed6e service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:53.145008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b7bec-4fd2-4f12-87e7-ac543555ed6e service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:53.145274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b7bec-4fd2-4f12-87e7-ac543555ed6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:53.146992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-065b7bec-4fd2-4f12-87e7-ac543555ed6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:53.158092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b7bec-4fd2-4f12-87e7-ac543555ed6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:53.159682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b7bec-4fd2-4f12-87e7-ac543555ed6e service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:53.180648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1223/2453] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:57.533040 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c177a-16c3-4c6f-a96e-502e3b991b09 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:57.542507 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c177a-16c3-4c6f-a96e-502e3b991b09 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:57.546409 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:57.547034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:57.547315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:57.547589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:57.549325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:57.612941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:57.614614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:57.625635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1231/2454] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:34:58.534958 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b7bec-4fd2-4f12-87e7-ac543555ed6e service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:58.546290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b7bec-4fd2-4f12-87e7-ac543555ed6e service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:58.552083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:58.552684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:58.552963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:58.553234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:58.555050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:58.653154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:58.654829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:58.684624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1224/2455] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:35:05.865354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:05.865354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:05.934269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f7103-8b9d-4be9-9312-e64402ae76c6 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:05.939376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd156c71-dacf-45e1-abe4-751704e8ce55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:05.939982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd156c71-dacf-45e1-abe4-751704e8ce55 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:05.940267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd156c71-dacf-45e1-abe4-751704e8ce55 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:05.940524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd156c71-dacf-45e1-abe4-751704e8ce55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:05.942310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dd156c71-dacf-45e1-abe4-751704e8ce55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:05.953898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd156c71-dacf-45e1-abe4-751704e8ce55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:05.955563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd156c71-dacf-45e1-abe4-751704e8ce55 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:05.965618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1232/2456] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:35:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:35:05.987543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:05.989148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:06.013259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a24b7d-c1fe-4df8-be26-56262785d2c2 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:06.018165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:06.018915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:06.019234 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:06.019526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] PATH_INFO: `/v3/users/b1971caade1b4c34a6c69c600070bc5e` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:06.020655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1971caade1b4c34a6c69c600070bc5e)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:06.022670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:06.228671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1971caade1b4c34a6c69c600070bc5e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.229261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1971caade1b4c34a6c69c600070bc5e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.229966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1971caade1b4c34a6c69c600070bc5e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.305727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1971caade1b4c34a6c69c600070bc5e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.313400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1971caade1b4c34a6c69c600070bc5e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.330922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1225/2457] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:35:05 2022] DELETE /identity/v3/users/b1971caade1b4c34a6c69c600070bc5e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:35:06.576558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd156c71-dacf-45e1-abe4-751704e8ce55 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:06.590903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd156c71-dacf-45e1-abe4-751704e8ce55 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:06.648925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:06.649541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:06.649817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:06.650116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] PATH_INFO: `/v3/projects/2ac977e5066e413c9e9cc06b5a012fdf` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:06.651227 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac977e5066e413c9e9cc06b5a012fdf)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:06.653143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:06.759162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:06.776843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac977e5066e413c9e9cc06b5a012fdf', 'request_id': 'req-974e9a16-8968-406e-b5e9-f294bd2632a5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.782554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac977e5066e413c9e9cc06b5a012fdf', 'request_id': 'req-974e9a16-8968-406e-b5e9-f294bd2632a5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.854332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac977e5066e413c9e9cc06b5a012fdf', 'request_id': 'req-974e9a16-8968-406e-b5e9-f294bd2632a5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.866304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] The token cache is being invalidate because project 2ac977e5066e413c9e9cc06b5a012fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:35:06.866911 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.867341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:06.868907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1233/2458] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:35:06 2022] DELETE /identity/v3/projects/2ac977e5066e413c9e9cc06b5a012fdf => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:06.873018 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:06.874574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:06.888008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f9c6a-93f6-4773-8e07-74bf8398e09c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:06.905466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:06.945846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:06.946504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:06.946785 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:06.947069 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] PATH_INFO: `/v3/users/f0f583136b2c48eda6308f6ce74b5a92` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:06.948201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f583136b2c48eda6308f6ce74b5a92)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:06.950305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:07.175920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f583136b2c48eda6308f6ce74b5a92'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:07.176569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f583136b2c48eda6308f6ce74b5a92'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:07.177074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f583136b2c48eda6308f6ce74b5a92'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:07.276851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f583136b2c48eda6308f6ce74b5a92'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:07.284455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f583136b2c48eda6308f6ce74b5a92'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:07.302338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1226/2459] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:35:06 2022] DELETE /identity/v3/users/f0f583136b2c48eda6308f6ce74b5a92 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:07.798082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:07.799589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:07.815310 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e9a16-8968-406e-b5e9-f294bd2632a5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:07.877428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:07.878138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:07.878378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:07.882590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] PATH_INFO: `/v3/projects/7047ba5e9e034ab39ddf41e0fa2224ab` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:07.882590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7047ba5e9e034ab39ddf41e0fa2224ab)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:07.882590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:07.986616 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7047ba5e9e034ab39ddf41e0fa2224ab', 'request_id': 'req-9b0b44a1-008a-417e-a517-69baea730a12'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:07.992184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7047ba5e9e034ab39ddf41e0fa2224ab', 'request_id': 'req-9b0b44a1-008a-417e-a517-69baea730a12'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:08.088700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7047ba5e9e034ab39ddf41e0fa2224ab', 'request_id': 'req-9b0b44a1-008a-417e-a517-69baea730a12'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:08.101036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] The token cache is being invalidate because project 7047ba5e9e034ab39ddf41e0fa2224ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:35:08.101348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:08.102020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:08.103497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1234/2460] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:35:07 2022] DELETE /identity/v3/projects/7047ba5e9e034ab39ddf41e0fa2224ab => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:17.298751 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=44,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:18.097785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=13,keystone:SELECT=65,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:40.379662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:40.381076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:40.442111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253c34d-a845-4122-a863-43dd4012d0e2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:40.491189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da395ad-2072-40ff-a264-707df01f7348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:40.491454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da395ad-2072-40ff-a264-707df01f7348 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:40.491725 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da395ad-2072-40ff-a264-707df01f7348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:40.492001 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da395ad-2072-40ff-a264-707df01f7348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:40.493835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9da395ad-2072-40ff-a264-707df01f7348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:40.558537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da395ad-2072-40ff-a264-707df01f7348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:40.560570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da395ad-2072-40ff-a264-707df01f7348 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:40.570551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1227/2461] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:35:44.420708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:44.422274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:44.437378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b44a1-008a-417e-a517-69baea730a12 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:44.442114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:44.442740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:44.443019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:44.443292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] PATH_INFO: `/v3/users/cd79bc9c3a694fe2a4e3c645e682e438` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:44.444428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd79bc9c3a694fe2a4e3c645e682e438)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:44.446488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:44.673341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd79bc9c3a694fe2a4e3c645e682e438'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:44.674062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd79bc9c3a694fe2a4e3c645e682e438'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:44.674736 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd79bc9c3a694fe2a4e3c645e682e438'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:44.683297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd79bc9c3a694fe2a4e3c645e682e438'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:44.702493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6271dfa3-f9cd-4a63-94ac-bc86464756dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd79bc9c3a694fe2a4e3c645e682e438'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:44.822442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1235/2462] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:35:44 2022] DELETE /identity/v3/users/cd79bc9c3a694fe2a4e3c645e682e438 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:45.142045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da395ad-2072-40ff-a264-707df01f7348 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:45.154026 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da395ad-2072-40ff-a264-707df01f7348 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:45.218808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:45.219574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:45.220042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:45.220370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] PATH_INFO: `/v3/projects/86e8de8aeabf4b8fa67ed0784fc39e46` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:45.221648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e8de8aeabf4b8fa67ed0784fc39e46)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:45.223674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:45.232452 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:45.407797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e8de8aeabf4b8fa67ed0784fc39e46', 'request_id': 'req-f977f6e1-095c-4eab-9f41-f40819ac7eb3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:45.413236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e8de8aeabf4b8fa67ed0784fc39e46', 'request_id': 'req-f977f6e1-095c-4eab-9f41-f40819ac7eb3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:45.420347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e8de8aeabf4b8fa67ed0784fc39e46', 'request_id': 'req-f977f6e1-095c-4eab-9f41-f40819ac7eb3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:45.563094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] The token cache is being invalidate because project 86e8de8aeabf4b8fa67ed0784fc39e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:35:45.563469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:45.564028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f977f6e1-095c-4eab-9f41-f40819ac7eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:45.565631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1228/2463] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:35:45 2022] DELETE /identity/v3/projects/86e8de8aeabf4b8fa67ed0784fc39e46 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:45.571576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9013c3c-9e0f-4448-8bea-67a1df690fe0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:45.572088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9013c3c-9e0f-4448-8bea-67a1df690fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:45.572088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9013c3c-9e0f-4448-8bea-67a1df690fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:45.591515 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a9013c3c-9e0f-4448-8bea-67a1df690fe0 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:45.592721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a9013c3c-9e0f-4448-8bea-67a1df690fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:45.651663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1236/2464] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:45.660642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70481fc2-c6ed-4720-8422-98ea00cae96a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:45.660926 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70481fc2-c6ed-4720-8422-98ea00cae96a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:45.661206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70481fc2-c6ed-4720-8422-98ea00cae96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:45.681379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-70481fc2-c6ed-4720-8422-98ea00cae96a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:45.682634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-70481fc2-c6ed-4720-8422-98ea00cae96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:45.690597 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1229/2465] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:45.694931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9013c3c-9e0f-4448-8bea-67a1df690fe0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:45.705744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9013c3c-9e0f-4448-8bea-67a1df690fe0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:45.710533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c778c-8055-43da-a5c2-9089592bb068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:45.711185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c778c-8055-43da-a5c2-9089592bb068 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:45.711459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c778c-8055-43da-a5c2-9089592bb068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:45.719526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c778c-8055-43da-a5c2-9089592bb068 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:45.719526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c778c-8055-43da-a5c2-9089592bb068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:35:45.719526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c778c-8055-43da-a5c2-9089592bb068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:45.719526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c778c-8055-43da-a5c2-9089592bb068 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:45.726291 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1237/2466] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:35:45.730680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70481fc2-c6ed-4720-8422-98ea00cae96a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:45.740269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70481fc2-c6ed-4720-8422-98ea00cae96a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:45.745491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0cf9-a003-4702-b85d-dd0bcf341360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:45.746115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0cf9-a003-4702-b85d-dd0bcf341360 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:45.746402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0cf9-a003-4702-b85d-dd0bcf341360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:45.746679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0cf9-a003-4702-b85d-dd0bcf341360 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:45.747322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d0cf9-a003-4702-b85d-dd0bcf341360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:45.749276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d0cf9-a003-4702-b85d-dd0bcf341360 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:45.881117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1230/2467] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:35:45 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:35:45.882324 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c778c-8055-43da-a5c2-9089592bb068 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:45.892987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c778c-8055-43da-a5c2-9089592bb068 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:45.952603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654985da-c74c-46e7-b27c-5eb31949cde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:45.953208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654985da-c74c-46e7-b27c-5eb31949cde6 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:45.953489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654985da-c74c-46e7-b27c-5eb31949cde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:45.953761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654985da-c74c-46e7-b27c-5eb31949cde6 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:45.954419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654985da-c74c-46e7-b27c-5eb31949cde6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:45.956352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654985da-c74c-46e7-b27c-5eb31949cde6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:45.964266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:46.091916 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1238/2468] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:35:45 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:35:46.096052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0cf9-a003-4702-b85d-dd0bcf341360 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:46.107007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0cf9-a003-4702-b85d-dd0bcf341360 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:46.111779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d9489-303d-4920-af03-37752430dd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:46.112513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d9489-303d-4920-af03-37752430dd24 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:46.112797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d9489-303d-4920-af03-37752430dd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:46.113072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d9489-303d-4920-af03-37752430dd24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:46.113724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d9489-303d-4920-af03-37752430dd24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:46.115406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d9489-303d-4920-af03-37752430dd24 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:46.127037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1231/2469] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:35:46.131297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654985da-c74c-46e7-b27c-5eb31949cde6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:46.142304 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654985da-c74c-46e7-b27c-5eb31949cde6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:46.147171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbe9f2-9ca5-413f-a296-50895b216aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:46.147780 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbe9f2-9ca5-413f-a296-50895b216aef admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:46.148043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbe9f2-9ca5-413f-a296-50895b216aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:46.148338 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbe9f2-9ca5-413f-a296-50895b216aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:46.148995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbe9f2-9ca5-413f-a296-50895b216aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:46.150657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbe9f2-9ca5-413f-a296-50895b216aef admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:46.162985 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1239/2470] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:35:46.167848 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d9489-303d-4920-af03-37752430dd24 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:46.178376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d9489-303d-4920-af03-37752430dd24 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:46.183735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8b9df-414f-4173-8cf5-174b62ba86e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:46.184382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8b9df-414f-4173-8cf5-174b62ba86e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:46.184660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8b9df-414f-4173-8cf5-174b62ba86e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:46.184932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8b9df-414f-4173-8cf5-174b62ba86e2 admin admin] PATH_INFO: `/v3/projects/37264cb6f978410c916ac0b42314333e/users/5efb42e49d1a4d3c949f181940a0e02c/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:46.199861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8b9df-414f-4173-8cf5-174b62ba86e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37264cb6f978410c916ac0b42314333e, user_id=5efb42e49d1a4d3c949f181940a0e02c, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:46.202593 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8b9df-414f-4173-8cf5-174b62ba86e2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:46.329513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1232/2471] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:35:46 2022] PUT /identity/v3/projects/37264cb6f978410c916ac0b42314333e/users/5efb42e49d1a4d3c949f181940a0e02c/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:46.340207 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56a862-5064-45eb-a589-68059d94919f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:46.340488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56a862-5064-45eb-a589-68059d94919f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:46.340760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56a862-5064-45eb-a589-68059d94919f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:46.368636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2e56a862-5064-45eb-a589-68059d94919f None None] MFA Rules not processed for user `5efb42e49d1a4d3c949f181940a0e02c`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:46.369979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2e56a862-5064-45eb-a589-68059d94919f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:46.438051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1240/2472] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:35:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:46.442737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8b9df-414f-4173-8cf5-174b62ba86e2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:46.506258 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8b9df-414f-4173-8cf5-174b62ba86e2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:46.510955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec78e0-85ed-451f-bada-31aa9ee08d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:46.511547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec78e0-85ed-451f-bada-31aa9ee08d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:46.511824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec78e0-85ed-451f-bada-31aa9ee08d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:46.512346 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec78e0-85ed-451f-bada-31aa9ee08d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:46.512972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec78e0-85ed-451f-bada-31aa9ee08d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:46.514912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec78e0-85ed-451f-bada-31aa9ee08d5a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:46.621343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1233/2473] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:35:46 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:35:46.625468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56a862-5064-45eb-a589-68059d94919f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:46.637326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56a862-5064-45eb-a589-68059d94919f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:46.696091 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db74aaf-6aa0-408b-9038-c0360d486212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:46.696707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db74aaf-6aa0-408b-9038-c0360d486212 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:46.696987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db74aaf-6aa0-408b-9038-c0360d486212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:46.697287 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db74aaf-6aa0-408b-9038-c0360d486212 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:46.697941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db74aaf-6aa0-408b-9038-c0360d486212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:46.699890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db74aaf-6aa0-408b-9038-c0360d486212 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:46.785497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1241/2474] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:35:46 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:35:46.789563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec78e0-85ed-451f-bada-31aa9ee08d5a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:46.800164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec78e0-85ed-451f-bada-31aa9ee08d5a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:46.804854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daecb18-4591-4279-b6f1-0f2ce6da9261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:46.805446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daecb18-4591-4279-b6f1-0f2ce6da9261 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:46.805723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daecb18-4591-4279-b6f1-0f2ce6da9261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:46.806050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daecb18-4591-4279-b6f1-0f2ce6da9261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:46.806713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daecb18-4591-4279-b6f1-0f2ce6da9261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:46.808309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daecb18-4591-4279-b6f1-0f2ce6da9261 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:46.810507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:46.819325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1234/2475] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:35:46.824084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db74aaf-6aa0-408b-9038-c0360d486212 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:46.834019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db74aaf-6aa0-408b-9038-c0360d486212 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:46.839165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd3a72-5fd3-4569-8aa1-cbb5974f8474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:46.839810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd3a72-5fd3-4569-8aa1-cbb5974f8474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:46.840169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd3a72-5fd3-4569-8aa1-cbb5974f8474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:46.840440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd3a72-5fd3-4569-8aa1-cbb5974f8474 admin admin] PATH_INFO: `/v3/projects/1fba3ec7fd4a4d5cadfdac3989a29112/users/21800ed035e543f5867fe910c955352e/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:46.854989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd3a72-5fd3-4569-8aa1-cbb5974f8474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fba3ec7fd4a4d5cadfdac3989a29112, user_id=21800ed035e543f5867fe910c955352e, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:46.857701 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd3a72-5fd3-4569-8aa1-cbb5974f8474 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:46.973818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1242/2476] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:35:46 2022] PUT /identity/v3/projects/1fba3ec7fd4a4d5cadfdac3989a29112/users/21800ed035e543f5867fe910c955352e/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:35:46.986742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daecb18-4591-4279-b6f1-0f2ce6da9261 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:46.988238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daecb18-4591-4279-b6f1-0f2ce6da9261 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:47.048498 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac608c5-62d4-45ae-b47f-2df74b7ecbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:47.049117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac608c5-62d4-45ae-b47f-2df74b7ecbbd admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:47.049406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac608c5-62d4-45ae-b47f-2df74b7ecbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:47.049678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac608c5-62d4-45ae-b47f-2df74b7ecbbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:47.050401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac608c5-62d4-45ae-b47f-2df74b7ecbbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:47.052000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac608c5-62d4-45ae-b47f-2df74b7ecbbd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:47.062706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1235/2477] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:35:47.066952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd3a72-5fd3-4569-8aa1-cbb5974f8474 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:47.078026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd3a72-5fd3-4569-8aa1-cbb5974f8474 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:47.083043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232d355-f029-4940-8259-7439e01dd8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:47.083709 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232d355-f029-4940-8259-7439e01dd8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:47.083988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232d355-f029-4940-8259-7439e01dd8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:47.084268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232d355-f029-4940-8259-7439e01dd8c8 admin admin] PATH_INFO: `/v3/projects/1fba3ec7fd4a4d5cadfdac3989a29112/users/21800ed035e543f5867fe910c955352e/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:47.086384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232d355-f029-4940-8259-7439e01dd8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fba3ec7fd4a4d5cadfdac3989a29112, user_id=21800ed035e543f5867fe910c955352e, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:47.089057 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232d355-f029-4940-8259-7439e01dd8c8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:47.099904 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1243/2478] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:35:47 2022] PUT /identity/v3/projects/1fba3ec7fd4a4d5cadfdac3989a29112/users/21800ed035e543f5867fe910c955352e/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:47.110225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4847c5-cf08-4364-82df-07925ce01947 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:47.110514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4847c5-cf08-4364-82df-07925ce01947 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:47.110791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4847c5-cf08-4364-82df-07925ce01947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:47.139906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6a4847c5-cf08-4364-82df-07925ce01947 None None] MFA Rules not processed for user `21800ed035e543f5867fe910c955352e`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:47.141126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6a4847c5-cf08-4364-82df-07925ce01947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:47.210456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1236/2479] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:35:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:35:47.221586 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232d355-f029-4940-8259-7439e01dd8c8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:47.223474 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b232d355-f029-4940-8259-7439e01dd8c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:47.247246 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:47.323144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232d355-f029-4940-8259-7439e01dd8c8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:47.327827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9a72b-482c-4e71-907f-36808cb281fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:47.328180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a72b-482c-4e71-907f-36808cb281fa service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:47.328463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a72b-482c-4e71-907f-36808cb281fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:47.328737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a72b-482c-4e71-907f-36808cb281fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:47.339433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9a72b-482c-4e71-907f-36808cb281fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:47.340662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9a72b-482c-4e71-907f-36808cb281fa service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:47.351062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1244/2480] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:35:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:35:47.634496 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4847c5-cf08-4364-82df-07925ce01947 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:47.644811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4847c5-cf08-4364-82df-07925ce01947 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:47.648691 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e30e5-00eb-422d-9adb-ae3d64d55970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:47.649296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e30e5-00eb-422d-9adb-ae3d64d55970 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:47.649572 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e30e5-00eb-422d-9adb-ae3d64d55970 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:47.649851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e30e5-00eb-422d-9adb-ae3d64d55970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:47.707767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e30e5-00eb-422d-9adb-ae3d64d55970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:47.709266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e30e5-00eb-422d-9adb-ae3d64d55970 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:47.718437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1237/2481] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:35:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:35:53.345644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166bcf06-34b7-46b3-b470-679cec817dcd None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:53.346123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166bcf06-34b7-46b3-b470-679cec817dcd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:53.346217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166bcf06-34b7-46b3-b470-679cec817dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:53.406390 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-166bcf06-34b7-46b3-b470-679cec817dcd None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:53.407655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-166bcf06-34b7-46b3-b470-679cec817dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:53.475670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1245/2482] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:35:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:53.701266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e30e5-00eb-422d-9adb-ae3d64d55970 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:53.702749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ac1e30e5-00eb-422d-9adb-ae3d64d55970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:53.725131 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:53.794833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e30e5-00eb-422d-9adb-ae3d64d55970 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:53.799993 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c431c-3068-4c21-af36-4da78315e930 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:53.800598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c431c-3068-4c21-af36-4da78315e930 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:53.800876 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c431c-3068-4c21-af36-4da78315e930 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:53.801152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c431c-3068-4c21-af36-4da78315e930 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:53.809747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c431c-3068-4c21-af36-4da78315e930 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:53.811445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c431c-3068-4c21-af36-4da78315e930 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:53.819794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1238/2483] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:35:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:36:00.529675 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166bcf06-34b7-46b3-b470-679cec817dcd None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:00.531539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-166bcf06-34b7-46b3-b470-679cec817dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:00.589151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166bcf06-34b7-46b3-b470-679cec817dcd None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:00.592770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaed54-16a0-4f46-a788-865170a43de7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:00.593373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaed54-16a0-4f46-a788-865170a43de7 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:00.593653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaed54-16a0-4f46-a788-865170a43de7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:00.593951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaed54-16a0-4f46-a788-865170a43de7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:00.595695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fccaed54-16a0-4f46-a788-865170a43de7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:00.605935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaed54-16a0-4f46-a788-865170a43de7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:00.607375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaed54-16a0-4f46-a788-865170a43de7 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:00.615184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1246/2484] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:03.821137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=33 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:05.232642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c431c-3068-4c21-af36-4da78315e930 service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:05.304649 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c431c-3068-4c21-af36-4da78315e930 service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:05.311178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:05.312144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:05.312515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:05.312893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] PATH_INFO: `/v3/users/5efb42e49d1a4d3c949f181940a0e02c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:05.314173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=5efb42e49d1a4d3c949f181940a0e02c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:05.316250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:05.670259 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5efb42e49d1a4d3c949f181940a0e02c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:05.671107 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5efb42e49d1a4d3c949f181940a0e02c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:05.671523 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5efb42e49d1a4d3c949f181940a0e02c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:05.756022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5efb42e49d1a4d3c949f181940a0e02c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:05.764191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5efb42e49d1a4d3c949f181940a0e02c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:05.786619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1239/2485] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:05 2022] DELETE /identity/v3/users/5efb42e49d1a4d3c949f181940a0e02c => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:05.799955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c902c-f79d-4523-8ed0-d4965396803d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:05.800528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c902c-f79d-4523-8ed0-d4965396803d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:05.800528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c902c-f79d-4523-8ed0-d4965396803d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:05.819262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ce2c902c-f79d-4523-8ed0-d4965396803d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:05.820470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ce2c902c-f79d-4523-8ed0-d4965396803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:05.832143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:05.883274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1247/2486] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:05.891035 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:05.892473 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:05.950075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3241c-e96d-4bc2-91a9-5e35614d9bda admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:05.953627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5179495-97aa-4033-b844-7fa1838da636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:05.954263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5179495-97aa-4033-b844-7fa1838da636 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:05.954548 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5179495-97aa-4033-b844-7fa1838da636 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:05.956560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5179495-97aa-4033-b844-7fa1838da636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:05.964513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5179495-97aa-4033-b844-7fa1838da636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:05.965970 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5179495-97aa-4033-b844-7fa1838da636 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:05.976061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1240/2487] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:06.495115 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c902c-f79d-4523-8ed0-d4965396803d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:06.496583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ce2c902c-f79d-4523-8ed0-d4965396803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:06.509266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c902c-f79d-4523-8ed0-d4965396803d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:06.514732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:06.515379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:06.515657 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:06.515934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] PATH_INFO: `/v3/projects/37264cb6f978410c916ac0b42314333e` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:06.517120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37264cb6f978410c916ac0b42314333e)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:06.519254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:06.648475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37264cb6f978410c916ac0b42314333e', 'request_id': 'req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.654100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37264cb6f978410c916ac0b42314333e', 'request_id': 'req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.753611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37264cb6f978410c916ac0b42314333e', 'request_id': 'req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.767905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] The token cache is being invalidate because project 37264cb6f978410c916ac0b42314333e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:06.768423 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.769477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.771059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1248/2488] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:06 2022] DELETE /identity/v3/projects/37264cb6f978410c916ac0b42314333e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:06.775913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5179495-97aa-4033-b844-7fa1838da636 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:06.786486 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5179495-97aa-4033-b844-7fa1838da636 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:06.805411 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:06.844450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:06.845081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:06.845359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:06.845632 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] PATH_INFO: `/v3/users/21800ed035e543f5867fe910c955352e` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:06.846829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=21800ed035e543f5867fe910c955352e)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:06.848860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:07.184192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21800ed035e543f5867fe910c955352e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:07.184852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21800ed035e543f5867fe910c955352e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:07.185462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21800ed035e543f5867fe910c955352e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:07.267605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21800ed035e543f5867fe910c955352e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:07.276082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-737d7109-654b-4ae7-8be5-61018de4e3bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21800ed035e543f5867fe910c955352e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:07.296747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1241/2489] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:06 2022] DELETE /identity/v3/users/21800ed035e543f5867fe910c955352e => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:07.938153 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:07.939679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:07.953818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cbd19-2bdf-4f72-8aec-3bb83aa76949 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:08.013219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:08.013776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:08.014088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:08.014367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] PATH_INFO: `/v3/projects/1fba3ec7fd4a4d5cadfdac3989a29112` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:08.015497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fba3ec7fd4a4d5cadfdac3989a29112)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:08.017480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:08.171202 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:08.174962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fba3ec7fd4a4d5cadfdac3989a29112', 'request_id': 'req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.180425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fba3ec7fd4a4d5cadfdac3989a29112', 'request_id': 'req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.330202 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fba3ec7fd4a4d5cadfdac3989a29112', 'request_id': 'req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.348502 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] The token cache is being invalidate because project 1fba3ec7fd4a4d5cadfdac3989a29112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:08.348795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.349517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6cfd1297-e5eb-4f81-b6c5-4d1cfce64c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.351716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1249/2490] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:07 2022] DELETE /identity/v3/projects/1fba3ec7fd4a4d5cadfdac3989a29112 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:08.356840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd852e0-a3e3-41be-a173-448b64fc333c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:08.357355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd852e0-a3e3-41be-a173-448b64fc333c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:08.357452 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd852e0-a3e3-41be-a173-448b64fc333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:08.377791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-bcd852e0-a3e3-41be-a173-448b64fc333c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:08.383120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bcd852e0-a3e3-41be-a173-448b64fc333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:08.443414 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1242/2491] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:08.451844 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b04aa2-106b-4ed2-8de6-3028f7dc221c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:08.452122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b04aa2-106b-4ed2-8de6-3028f7dc221c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:08.452396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b04aa2-106b-4ed2-8de6-3028f7dc221c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:08.472856 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-30b04aa2-106b-4ed2-8de6-3028f7dc221c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:08.474095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-30b04aa2-106b-4ed2-8de6-3028f7dc221c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:08.482201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1250/2492] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:08.487645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd852e0-a3e3-41be-a173-448b64fc333c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:08.498738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd852e0-a3e3-41be-a173-448b64fc333c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:08.503878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee769fa0-1b71-46a4-9e6a-8b81901f1618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:08.504497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee769fa0-1b71-46a4-9e6a-8b81901f1618 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:08.504782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee769fa0-1b71-46a4-9e6a-8b81901f1618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:08.505060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee769fa0-1b71-46a4-9e6a-8b81901f1618 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:08.505761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee769fa0-1b71-46a4-9e6a-8b81901f1618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:08.513801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee769fa0-1b71-46a4-9e6a-8b81901f1618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:08.513801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee769fa0-1b71-46a4-9e6a-8b81901f1618 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:08.520979 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1243/2493] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:36:08.526100 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b04aa2-106b-4ed2-8de6-3028f7dc221c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:08.535603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b04aa2-106b-4ed2-8de6-3028f7dc221c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:08.540395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6296937-ad07-4184-a5d9-0365de281267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:08.541108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6296937-ad07-4184-a5d9-0365de281267 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:08.541397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6296937-ad07-4184-a5d9-0365de281267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:08.541668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6296937-ad07-4184-a5d9-0365de281267 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:08.542327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6296937-ad07-4184-a5d9-0365de281267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:08.544402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6296937-ad07-4184-a5d9-0365de281267 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:08.692761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1251/2494] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:08 2022] POST /identity/v3/projects => generated 363 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:08.697263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee769fa0-1b71-46a4-9e6a-8b81901f1618 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:08.707793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee769fa0-1b71-46a4-9e6a-8b81901f1618 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:08.735806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:08.769164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0e393-33e8-4953-94d2-5906f4000ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:08.769916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0e393-33e8-4953-94d2-5906f4000ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:08.770079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0e393-33e8-4953-94d2-5906f4000ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:08.770355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0e393-33e8-4953-94d2-5906f4000ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:08.771005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0e393-33e8-4953-94d2-5906f4000ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:08.773067 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0e393-33e8-4953-94d2-5906f4000ff6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:08.910555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1244/2495] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:08 2022] POST /identity/v3/users => generated 282 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:08.914755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6296937-ad07-4184-a5d9-0365de281267 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:08.926348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6296937-ad07-4184-a5d9-0365de281267 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:08.931832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d4c4d-c897-4a4b-906a-dcea38273c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:08.932459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d4c4d-c897-4a4b-906a-dcea38273c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:08.932742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d4c4d-c897-4a4b-906a-dcea38273c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:08.933017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d4c4d-c897-4a4b-906a-dcea38273c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:08.933705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d4c4d-c897-4a4b-906a-dcea38273c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:08.935369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d4c4d-c897-4a4b-906a-dcea38273c4c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:08.947618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1252/2496] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:08.951942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0e393-33e8-4953-94d2-5906f4000ff6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:08.962957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0e393-33e8-4953-94d2-5906f4000ff6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:08.967783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f907a-c064-4ea5-91b2-874af381ecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:08.968401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f907a-c064-4ea5-91b2-874af381ecd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:08.968683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f907a-c064-4ea5-91b2-874af381ecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:08.968962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f907a-c064-4ea5-91b2-874af381ecd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:08.969634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f907a-c064-4ea5-91b2-874af381ecd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:08.971285 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f907a-c064-4ea5-91b2-874af381ecd6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:08.984789 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1245/2497] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:08.989196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d4c4d-c897-4a4b-906a-dcea38273c4c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:09.000673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d4c4d-c897-4a4b-906a-dcea38273c4c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:09.005655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e3e2c-84a5-4bcf-9063-2b986cc4ec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:09.006510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e3e2c-84a5-4bcf-9063-2b986cc4ec36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:09.006822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e3e2c-84a5-4bcf-9063-2b986cc4ec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:09.007098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e3e2c-84a5-4bcf-9063-2b986cc4ec36 admin admin] PATH_INFO: `/v3/projects/38b4589b73a8453fb62170cf25975a0c/users/ce9f38bd47b648cc8d8cbe36902611e9/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:09.025506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e3e2c-84a5-4bcf-9063-2b986cc4ec36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b4589b73a8453fb62170cf25975a0c, user_id=ce9f38bd47b648cc8d8cbe36902611e9, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:09.028253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e3e2c-84a5-4bcf-9063-2b986cc4ec36 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:09.201812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1253/2498] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:08 2022] PUT /identity/v3/projects/38b4589b73a8453fb62170cf25975a0c/users/ce9f38bd47b648cc8d8cbe36902611e9/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:09.212236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28261b47-6d16-451e-ad1f-5458a83154d5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:09.212520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28261b47-6d16-451e-ad1f-5458a83154d5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:09.212797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28261b47-6d16-451e-ad1f-5458a83154d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:09.242840 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-28261b47-6d16-451e-ad1f-5458a83154d5 None None] MFA Rules not processed for user `ce9f38bd47b648cc8d8cbe36902611e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:09.244062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-28261b47-6d16-451e-ad1f-5458a83154d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:09.309065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1246/2499] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:09.313636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e3e2c-84a5-4bcf-9063-2b986cc4ec36 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:09.377807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e3e2c-84a5-4bcf-9063-2b986cc4ec36 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:09.382783 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef79c6-ddd4-4ddf-b531-5df57c29b45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:09.383420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef79c6-ddd4-4ddf-b531-5df57c29b45f admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:09.383704 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef79c6-ddd4-4ddf-b531-5df57c29b45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:09.383972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef79c6-ddd4-4ddf-b531-5df57c29b45f admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:09.384634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef79c6-ddd4-4ddf-b531-5df57c29b45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:09.386903 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef79c6-ddd4-4ddf-b531-5df57c29b45f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:09.546774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1254/2500] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:09 2022] POST /identity/v3/projects => generated 365 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:09.551178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28261b47-6d16-451e-ad1f-5458a83154d5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:09.562861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28261b47-6d16-451e-ad1f-5458a83154d5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:09.627461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240376a-29f4-4740-a634-401dd565d3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:09.628340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240376a-29f4-4740-a634-401dd565d3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:09.628631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240376a-29f4-4740-a634-401dd565d3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:09.628947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240376a-29f4-4740-a634-401dd565d3c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:09.629559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240376a-29f4-4740-a634-401dd565d3c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:09.631517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240376a-29f4-4740-a634-401dd565d3c4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:09.876152 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1247/2501] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:09 2022] POST /identity/v3/users => generated 289 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:09.877741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef79c6-ddd4-4ddf-b531-5df57c29b45f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:09.888971 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef79c6-ddd4-4ddf-b531-5df57c29b45f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:09.893606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5ba68-38d2-40cb-9c5c-e4314e94e43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:09.894235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5ba68-38d2-40cb-9c5c-e4314e94e43e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:09.894518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5ba68-38d2-40cb-9c5c-e4314e94e43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:09.894791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5ba68-38d2-40cb-9c5c-e4314e94e43e admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:09.895471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5ba68-38d2-40cb-9c5c-e4314e94e43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:09.897112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5ba68-38d2-40cb-9c5c-e4314e94e43e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:09.912454 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1255/2502] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:09.913714 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240376a-29f4-4740-a634-401dd565d3c4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:09.923931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240376a-29f4-4740-a634-401dd565d3c4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:09.928660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b6d53-c879-4d24-8d8a-91a3b9b728ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:09.929327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b6d53-c879-4d24-8d8a-91a3b9b728ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:09.929606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b6d53-c879-4d24-8d8a-91a3b9b728ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:09.929943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b6d53-c879-4d24-8d8a-91a3b9b728ef admin admin] PATH_INFO: `/v3/projects/47722fad88ed497bbcb2e79309aaff85/users/d3846a9a28374dcb89958d2540574b30/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:09.946419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b6d53-c879-4d24-8d8a-91a3b9b728ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=47722fad88ed497bbcb2e79309aaff85, user_id=d3846a9a28374dcb89958d2540574b30, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:09.949042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b6d53-c879-4d24-8d8a-91a3b9b728ef admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:10.050615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1248/2503] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:09 2022] PUT /identity/v3/projects/47722fad88ed497bbcb2e79309aaff85/users/d3846a9a28374dcb89958d2540574b30/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:10.054564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5ba68-38d2-40cb-9c5c-e4314e94e43e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:10.089668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5ba68-38d2-40cb-9c5c-e4314e94e43e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:10.108078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:10.124238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8257e33-f7d1-4fcc-b8fc-8590f5e601c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:10.124840 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8257e33-f7d1-4fcc-b8fc-8590f5e601c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.125116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8257e33-f7d1-4fcc-b8fc-8590f5e601c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.125397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8257e33-f7d1-4fcc-b8fc-8590f5e601c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.126065 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8257e33-f7d1-4fcc-b8fc-8590f5e601c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:10.127654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8257e33-f7d1-4fcc-b8fc-8590f5e601c7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:10.138625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1256/2504] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:10.143611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b6d53-c879-4d24-8d8a-91a3b9b728ef admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:10.149537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:10.154960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b6d53-c879-4d24-8d8a-91a3b9b728ef admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:10.159758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5f88e-007c-4d04-abfd-4f689a311284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:10.160413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5f88e-007c-4d04-abfd-4f689a311284 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.160700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5f88e-007c-4d04-abfd-4f689a311284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.160969 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5f88e-007c-4d04-abfd-4f689a311284 admin admin] PATH_INFO: `/v3/projects/47722fad88ed497bbcb2e79309aaff85/users/d3846a9a28374dcb89958d2540574b30/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.162992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5f88e-007c-4d04-abfd-4f689a311284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47722fad88ed497bbcb2e79309aaff85, user_id=d3846a9a28374dcb89958d2540574b30, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:10.165684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5f88e-007c-4d04-abfd-4f689a311284 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:10.176647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1249/2505] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:10 2022] PUT /identity/v3/projects/47722fad88ed497bbcb2e79309aaff85/users/d3846a9a28374dcb89958d2540574b30/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:10.188491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f49ac1-337a-429f-baf7-3e06d2c22e47 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.193046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f49ac1-337a-429f-baf7-3e06d2c22e47 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.193046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f49ac1-337a-429f-baf7-3e06d2c22e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.218953 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-95f49ac1-337a-429f-baf7-3e06d2c22e47 None None] MFA Rules not processed for user `d3846a9a28374dcb89958d2540574b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:10.220198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-95f49ac1-337a-429f-baf7-3e06d2c22e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:10.287874 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1257/2506] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:10.313110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5f88e-007c-4d04-abfd-4f689a311284 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:10.313110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ccb5f88e-007c-4d04-abfd-4f689a311284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:10.381782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5f88e-007c-4d04-abfd-4f689a311284 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:10.381782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e217169-e624-4f88-bda5-2e07d0e31125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:10.381782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217169-e624-4f88-bda5-2e07d0e31125 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.381782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217169-e624-4f88-bda5-2e07d0e31125 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.382875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217169-e624-4f88-bda5-2e07d0e31125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.382875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e217169-e624-4f88-bda5-2e07d0e31125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:10.382875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e217169-e624-4f88-bda5-2e07d0e31125 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:10.385972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1250/2507] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:11.679727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f49ac1-337a-429f-baf7-3e06d2c22e47 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:11.742762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f49ac1-337a-429f-baf7-3e06d2c22e47 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:11.747692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:11.748331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:11.748619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:11.748907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] PATH_INFO: `/v3/users/ce9f38bd47b648cc8d8cbe36902611e9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:11.750062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9f38bd47b648cc8d8cbe36902611e9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:11.752003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:12.000793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9f38bd47b648cc8d8cbe36902611e9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:12.001449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9f38bd47b648cc8d8cbe36902611e9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:12.002087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9f38bd47b648cc8d8cbe36902611e9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:12.008957 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9f38bd47b648cc8d8cbe36902611e9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:12.016113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:12.021615 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9f38bd47b648cc8d8cbe36902611e9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:12.135056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1258/2508] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:11 2022] DELETE /identity/v3/users/ce9f38bd47b648cc8d8cbe36902611e9 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:12.140094 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93425163-b5c1-4e7d-9ef5-62d24545c434 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:12.140558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93425163-b5c1-4e7d-9ef5-62d24545c434 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:12.140558 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93425163-b5c1-4e7d-9ef5-62d24545c434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:12.160554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-93425163-b5c1-4e7d-9ef5-62d24545c434 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:12.161766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-93425163-b5c1-4e7d-9ef5-62d24545c434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:12.220140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1251/2509] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:12.230558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:12.232041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:12.291295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a064e54-04cd-4f42-be59-8e371b25dcf9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:12.294929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863f582-a3c2-4d78-a8bd-22b16a9c1622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:12.295537 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863f582-a3c2-4d78-a8bd-22b16a9c1622 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:12.295822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863f582-a3c2-4d78-a8bd-22b16a9c1622 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:12.296097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863f582-a3c2-4d78-a8bd-22b16a9c1622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:12.305912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863f582-a3c2-4d78-a8bd-22b16a9c1622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:12.307421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863f582-a3c2-4d78-a8bd-22b16a9c1622 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:12.317102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1259/2510] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:12.771860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93425163-b5c1-4e7d-9ef5-62d24545c434 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:12.775250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-93425163-b5c1-4e7d-9ef5-62d24545c434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:12.785999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93425163-b5c1-4e7d-9ef5-62d24545c434 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:12.801695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:12.802617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:12.802995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:12.803366 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] PATH_INFO: `/v3/projects/38b4589b73a8453fb62170cf25975a0c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:12.804673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b4589b73a8453fb62170cf25975a0c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:12.806911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:12.979901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b4589b73a8453fb62170cf25975a0c', 'request_id': 'req-9328b4a6-9a52-4ffc-be27-eef726959807'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:12.985541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b4589b73a8453fb62170cf25975a0c', 'request_id': 'req-9328b4a6-9a52-4ffc-be27-eef726959807'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:12.991824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b4589b73a8453fb62170cf25975a0c', 'request_id': 'req-9328b4a6-9a52-4ffc-be27-eef726959807'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:13.106781 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] The token cache is being invalidate because project 38b4589b73a8453fb62170cf25975a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:13.107075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:13.107716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9328b4a6-9a52-4ffc-be27-eef726959807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:13.109229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1252/2511] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:12 2022] DELETE /identity/v3/projects/38b4589b73a8453fb62170cf25975a0c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:13.113564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863f582-a3c2-4d78-a8bd-22b16a9c1622 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:13.123857 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863f582-a3c2-4d78-a8bd-22b16a9c1622 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:13.146731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacad9e-ace1-485c-a27d-6dc285b53eef None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:13.147378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacad9e-ace1-485c-a27d-6dc285b53eef None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:13.147378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacad9e-ace1-485c-a27d-6dc285b53eef None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:13.149278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1253/2512] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:36:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:36:13.155143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24868b7-c59b-4eaf-861d-e61e36aa0b45 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:13.155634 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24868b7-c59b-4eaf-861d-e61e36aa0b45 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:13.155914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24868b7-c59b-4eaf-861d-e61e36aa0b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:13.188685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:13.193294 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:13.193968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:13.194255 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:13.194533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] PATH_INFO: `/v3/users/d3846a9a28374dcb89958d2540574b30` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:13.196280 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3846a9a28374dcb89958d2540574b30)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:13.198799 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d24868b7-c59b-4eaf-861d-e61e36aa0b45 None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:13.201448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:13.201684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d24868b7-c59b-4eaf-861d-e61e36aa0b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:13.281738 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1254/2513] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:36:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:13.293396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24868b7-c59b-4eaf-861d-e61e36aa0b45 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:13.294765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d24868b7-c59b-4eaf-861d-e61e36aa0b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:13.352772 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24868b7-c59b-4eaf-861d-e61e36aa0b45 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:13.356545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d17314-8a50-4bde-bf74-e43edd80782e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f214c2b77d4cbab2f15122ba21bf38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:13.357198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d17314-8a50-4bde-bf74-e43edd80782e service glance] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:13.357481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d17314-8a50-4bde-bf74-e43edd80782e service glance] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:13.357752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d17314-8a50-4bde-bf74-e43edd80782e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:13.368790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d17314-8a50-4bde-bf74-e43edd80782e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:13.370261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d17314-8a50-4bde-bf74-e43edd80782e service glance] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:13.381978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1255/2514] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:36:13.483669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3846a9a28374dcb89958d2540574b30'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:13.484322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3846a9a28374dcb89958d2540574b30'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:13.484921 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3846a9a28374dcb89958d2540574b30'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:13.492045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3846a9a28374dcb89958d2540574b30'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:13.504330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6a247686-aa6a-4de7-84f4-bfb10e66a70b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3846a9a28374dcb89958d2540574b30'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:13.653357 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1260/2515] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:13 2022] DELETE /identity/v3/users/d3846a9a28374dcb89958d2540574b30 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:14.174412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d17314-8a50-4bde-bf74-e43edd80782e service glance] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:14.175851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c7d17314-8a50-4bde-bf74-e43edd80782e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:14.190356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d17314-8a50-4bde-bf74-e43edd80782e service glance] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:14.262878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:14.263763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:14.264116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:14.264465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] PATH_INFO: `/v3/projects/47722fad88ed497bbcb2e79309aaff85` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:14.265741 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=47722fad88ed497bbcb2e79309aaff85)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:14.267948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:14.469641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:14.477075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47722fad88ed497bbcb2e79309aaff85', 'request_id': 'req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.482852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47722fad88ed497bbcb2e79309aaff85', 'request_id': 'req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.489246 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47722fad88ed497bbcb2e79309aaff85', 'request_id': 'req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.630333 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] The token cache is being invalidate because project 47722fad88ed497bbcb2e79309aaff85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:14.630630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.631278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fa2c78d6-3ee6-4708-aeed-6c3fb063bb1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.632817 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1256/2516] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:14 2022] DELETE /identity/v3/projects/47722fad88ed497bbcb2e79309aaff85 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:14.638126 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512164ab-485a-4b27-a4b7-e448123242a0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:14.638599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512164ab-485a-4b27-a4b7-e448123242a0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:14.638703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512164ab-485a-4b27-a4b7-e448123242a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:14.647343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:14.659468 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-512164ab-485a-4b27-a4b7-e448123242a0 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:14.660700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-512164ab-485a-4b27-a4b7-e448123242a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:14.722551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1261/2517] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:14.731244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95090d-0123-4c93-9f4a-81bea1c1c4e7 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:14.731527 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95090d-0123-4c93-9f4a-81bea1c1c4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:14.731804 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95090d-0123-4c93-9f4a-81bea1c1c4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:14.751612 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7b95090d-0123-4c93-9f4a-81bea1c1c4e7 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:14.752974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7b95090d-0123-4c93-9f4a-81bea1c1c4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:14.760930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1257/2518] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:14.765565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512164ab-485a-4b27-a4b7-e448123242a0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:14.775699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512164ab-485a-4b27-a4b7-e448123242a0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:14.780277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f872a2-98c8-4f7c-b8d7-14809bdceb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:14.780864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f872a2-98c8-4f7c-b8d7-14809bdceb53 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:14.781139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f872a2-98c8-4f7c-b8d7-14809bdceb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:14.781405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f872a2-98c8-4f7c-b8d7-14809bdceb53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:14.782082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f872a2-98c8-4f7c-b8d7-14809bdceb53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:14.782383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f872a2-98c8-4f7c-b8d7-14809bdceb53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:14.783995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f872a2-98c8-4f7c-b8d7-14809bdceb53 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:14.796121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1262/2519] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:36:14.800729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95090d-0123-4c93-9f4a-81bea1c1c4e7 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:14.810188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95090d-0123-4c93-9f4a-81bea1c1c4e7 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:14.814849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d4ce2-44d6-46e5-b5cc-b58c3ea0a961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:14.815446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d4ce2-44d6-46e5-b5cc-b58c3ea0a961 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:14.815734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d4ce2-44d6-46e5-b5cc-b58c3ea0a961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:14.816010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d4ce2-44d6-46e5-b5cc-b58c3ea0a961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:14.816616 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d4ce2-44d6-46e5-b5cc-b58c3ea0a961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:14.818570 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d4ce2-44d6-46e5-b5cc-b58c3ea0a961 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:15.018244 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1258/2520] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:14 2022] POST /identity/v3/projects => generated 373 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:15.022379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f872a2-98c8-4f7c-b8d7-14809bdceb53 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:15.032877 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f872a2-98c8-4f7c-b8d7-14809bdceb53 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:15.091199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c752bc-3d88-44c5-9db9-f7c9cdb140da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:15.091817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c752bc-3d88-44c5-9db9-f7c9cdb140da admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:15.092104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c752bc-3d88-44c5-9db9-f7c9cdb140da admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:15.092378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c752bc-3d88-44c5-9db9-f7c9cdb140da admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:15.093029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c752bc-3d88-44c5-9db9-f7c9cdb140da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:15.095186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c752bc-3d88-44c5-9db9-f7c9cdb140da admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:15.221014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1263/2521] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:15 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:15.222206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d4ce2-44d6-46e5-b5cc-b58c3ea0a961 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:15.239138 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d4ce2-44d6-46e5-b5cc-b58c3ea0a961 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:15.246012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a1ca1-3cea-4f57-8b45-593e3749f08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:15.246736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1ca1-3cea-4f57-8b45-593e3749f08f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:15.247065 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1ca1-3cea-4f57-8b45-593e3749f08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:15.247392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1ca1-3cea-4f57-8b45-593e3749f08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:15.248170 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a1ca1-3cea-4f57-8b45-593e3749f08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:15.250351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a1ca1-3cea-4f57-8b45-593e3749f08f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:15.263727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1259/2522] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:15.267823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c752bc-3d88-44c5-9db9-f7c9cdb140da admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:15.280114 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c752bc-3d88-44c5-9db9-f7c9cdb140da admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:15.284812 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf36bd1-90bd-4f02-b436-62bc0f0925d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:15.285421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36bd1-90bd-4f02-b436-62bc0f0925d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:15.285706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36bd1-90bd-4f02-b436-62bc0f0925d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:15.286039 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36bd1-90bd-4f02-b436-62bc0f0925d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:15.286711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf36bd1-90bd-4f02-b436-62bc0f0925d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:15.288346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf36bd1-90bd-4f02-b436-62bc0f0925d2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:15.299938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1264/2523] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:15.304066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a1ca1-3cea-4f57-8b45-593e3749f08f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:15.314492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a1ca1-3cea-4f57-8b45-593e3749f08f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:15.319101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c7250-b2a2-4514-b52f-99af0ffbf829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:15.319745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c7250-b2a2-4514-b52f-99af0ffbf829 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:15.320025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c7250-b2a2-4514-b52f-99af0ffbf829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:15.320830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c7250-b2a2-4514-b52f-99af0ffbf829 admin admin] PATH_INFO: `/v3/projects/8a5ddf23a4674d11a73e8323bae8f889/users/df551073b1d64ea9a8ce0e0207d6d498/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:15.336464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c7250-b2a2-4514-b52f-99af0ffbf829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5ddf23a4674d11a73e8323bae8f889, user_id=df551073b1d64ea9a8ce0e0207d6d498, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:15.339190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c7250-b2a2-4514-b52f-99af0ffbf829 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:15.519932 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1260/2524] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:15 2022] PUT /identity/v3/projects/8a5ddf23a4674d11a73e8323bae8f889/users/df551073b1d64ea9a8ce0e0207d6d498/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:36:15.530593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61308fb8-ab45-4eb4-8d46-6fb5ca63ec6f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:15.530825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61308fb8-ab45-4eb4-8d46-6fb5ca63ec6f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:15.531096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61308fb8-ab45-4eb4-8d46-6fb5ca63ec6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:15.559564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-61308fb8-ab45-4eb4-8d46-6fb5ca63ec6f None None] MFA Rules not processed for user `df551073b1d64ea9a8ce0e0207d6d498`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:15.560810 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-61308fb8-ab45-4eb4-8d46-6fb5ca63ec6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:15.624229 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:15.631104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1265/2525] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:15.639875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c7250-b2a2-4514-b52f-99af0ffbf829 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:15.641690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ac6c7250-b2a2-4514-b52f-99af0ffbf829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:15.698962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c7250-b2a2-4514-b52f-99af0ffbf829 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:15.702567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403948e-d60e-4c9f-99f6-f3939434b402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:15.703165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403948e-d60e-4c9f-99f6-f3939434b402 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:15.703441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403948e-d60e-4c9f-99f6-f3939434b402 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:15.703710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403948e-d60e-4c9f-99f6-f3939434b402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:15.712221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403948e-d60e-4c9f-99f6-f3939434b402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:15.713767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403948e-d60e-4c9f-99f6-f3939434b402 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:15.723217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1261/2526] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:18.487047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ec250-9f69-405f-bae1-6e8b4eb7552c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:18.487584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ec250-9f69-405f-bae1-6e8b4eb7552c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:18.487584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ec250-9f69-405f-bae1-6e8b4eb7552c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:18.508493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4b8ec250-9f69-405f-bae1-6e8b4eb7552c None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:18.509725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4b8ec250-9f69-405f-bae1-6e8b4eb7552c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:18.566755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1266/2527] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:36:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:18.579295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403948e-d60e-4c9f-99f6-f3939434b402 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:18.580711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1403948e-d60e-4c9f-99f6-f3939434b402 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:18.653104 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403948e-d60e-4c9f-99f6-f3939434b402 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:18.658370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2475c5-9049-48df-9bf4-f0b0168655a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:18.658984 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2475c5-9049-48df-9bf4-f0b0168655a5 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:18.659252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2475c5-9049-48df-9bf4-f0b0168655a5 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:18.659524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2475c5-9049-48df-9bf4-f0b0168655a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:18.666674 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:18.669247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2475c5-9049-48df-9bf4-f0b0168655a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:18.670903 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2475c5-9049-48df-9bf4-f0b0168655a5 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:18.679141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1262/2528] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:36:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:36:21.518003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ec250-9f69-405f-bae1-6e8b4eb7552c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:21.598317 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ec250-9f69-405f-bae1-6e8b4eb7552c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:21.608325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:21.608325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:21.608325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:21.608325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] PATH_INFO: `/v3/users/df551073b1d64ea9a8ce0e0207d6d498` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:21.608706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df551073b1d64ea9a8ce0e0207d6d498)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:21.608706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:21.794808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df551073b1d64ea9a8ce0e0207d6d498'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:21.795525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df551073b1d64ea9a8ce0e0207d6d498'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:21.796087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df551073b1d64ea9a8ce0e0207d6d498'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:21.805189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df551073b1d64ea9a8ce0e0207d6d498'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:21.817757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df551073b1d64ea9a8ce0e0207d6d498'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:21.908222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1267/2529] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:21 2022] DELETE /identity/v3/users/df551073b1d64ea9a8ce0e0207d6d498 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:36:21.914923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05d375-0d2f-42dd-bc9a-2b825fa103ac None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:21.915455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05d375-0d2f-42dd-bc9a-2b825fa103ac None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:21.915562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05d375-0d2f-42dd-bc9a-2b825fa103ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:21.935749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9f05d375-0d2f-42dd-bc9a-2b825fa103ac None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:21.937251 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9f05d375-0d2f-42dd-bc9a-2b825fa103ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:22.016736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1263/2530] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:22.027045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:22.028606 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:22.059096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:22.089168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979738df-fce7-45fc-9f79-2ad319a10fd2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:22.092940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49229466-fef8-4692-86c6-822af70f72fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:22.093551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49229466-fef8-4692-86c6-822af70f72fb service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:22.093834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49229466-fef8-4692-86c6-822af70f72fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:22.094146 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49229466-fef8-4692-86c6-822af70f72fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:22.104094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49229466-fef8-4692-86c6-822af70f72fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:22.105556 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49229466-fef8-4692-86c6-822af70f72fb service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:22.116403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1268/2531] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:22.681961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05d375-0d2f-42dd-bc9a-2b825fa103ac None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:22.683765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9f05d375-0d2f-42dd-bc9a-2b825fa103ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:22.698905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05d375-0d2f-42dd-bc9a-2b825fa103ac None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:22.703981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:22.704709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:22.705057 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:22.705438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] PATH_INFO: `/v3/projects/8a5ddf23a4674d11a73e8323bae8f889` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:22.727868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5ddf23a4674d11a73e8323bae8f889)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:22.730373 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:22.853907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5ddf23a4674d11a73e8323bae8f889', 'request_id': 'req-7c53dadb-2a13-4198-89b6-4c4386eabf90'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:22.859135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5ddf23a4674d11a73e8323bae8f889', 'request_id': 'req-7c53dadb-2a13-4198-89b6-4c4386eabf90'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:22.865859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5ddf23a4674d11a73e8323bae8f889', 'request_id': 'req-7c53dadb-2a13-4198-89b6-4c4386eabf90'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:22.975776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] The token cache is being invalidate because project 8a5ddf23a4674d11a73e8323bae8f889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:22.976471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:22.976773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c53dadb-2a13-4198-89b6-4c4386eabf90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:22.978340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1264/2532] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:22 2022] DELETE /identity/v3/projects/8a5ddf23a4674d11a73e8323bae8f889 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:22.983350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd07232-62b1-4304-9395-6b2b85f9c50e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:22.983882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd07232-62b1-4304-9395-6b2b85f9c50e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:22.983882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd07232-62b1-4304-9395-6b2b85f9c50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.005030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3cd07232-62b1-4304-9395-6b2b85f9c50e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:23.006306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3cd07232-62b1-4304-9395-6b2b85f9c50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:23.072867 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1269/2533] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:23.082426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f149b5-3569-482c-a1d9-6036d32cc42d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:23.082875 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f149b5-3569-482c-a1d9-6036d32cc42d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:23.083351 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f149b5-3569-482c-a1d9-6036d32cc42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.108976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-95f149b5-3569-482c-a1d9-6036d32cc42d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:23.110228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-95f149b5-3569-482c-a1d9-6036d32cc42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:23.118879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1265/2534] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:23.123083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd07232-62b1-4304-9395-6b2b85f9c50e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:23.133809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd07232-62b1-4304-9395-6b2b85f9c50e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:23.139023 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3a4e8-506e-451e-b475-7f53cd534953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:23.139640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3a4e8-506e-451e-b475-7f53cd534953 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:23.139924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3a4e8-506e-451e-b475-7f53cd534953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:23.140197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3a4e8-506e-451e-b475-7f53cd534953 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.140863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3a4e8-506e-451e-b475-7f53cd534953 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:23.141171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3a4e8-506e-451e-b475-7f53cd534953 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:23.142830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3a4e8-506e-451e-b475-7f53cd534953 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:23.157389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1270/2535] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:36:23.161456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f149b5-3569-482c-a1d9-6036d32cc42d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:23.172865 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f149b5-3569-482c-a1d9-6036d32cc42d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:23.177826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9797-152f-4cfb-b570-f72ab6ad16b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:23.178465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9797-152f-4cfb-b570-f72ab6ad16b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:23.178744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9797-152f-4cfb-b570-f72ab6ad16b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:23.179016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9797-152f-4cfb-b570-f72ab6ad16b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.180328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf9797-152f-4cfb-b570-f72ab6ad16b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:23.182538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf9797-152f-4cfb-b570-f72ab6ad16b5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:23.292991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1266/2536] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:23 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:23.297171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3a4e8-506e-451e-b475-7f53cd534953 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:23.309558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3a4e8-506e-451e-b475-7f53cd534953 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:23.372234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875ff7a-e984-4326-a092-f80bbfcc3c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:23.372845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875ff7a-e984-4326-a092-f80bbfcc3c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:23.373128 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875ff7a-e984-4326-a092-f80bbfcc3c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:23.373403 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875ff7a-e984-4326-a092-f80bbfcc3c08 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.374071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875ff7a-e984-4326-a092-f80bbfcc3c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:23.376678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875ff7a-e984-4326-a092-f80bbfcc3c08 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:23.469140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1271/2537] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:23 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:23.472935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9797-152f-4cfb-b570-f72ab6ad16b5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:23.484006 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9797-152f-4cfb-b570-f72ab6ad16b5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:23.489168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1053328-c823-4e81-aed0-9461c128b957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:23.489860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1053328-c823-4e81-aed0-9461c128b957 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:23.490280 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1053328-c823-4e81-aed0-9461c128b957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:23.490601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1053328-c823-4e81-aed0-9461c128b957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.491274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1053328-c823-4e81-aed0-9461c128b957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:23.493630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1053328-c823-4e81-aed0-9461c128b957 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:23.508598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1267/2538] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:23.510046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875ff7a-e984-4326-a092-f80bbfcc3c08 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:23.531182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875ff7a-e984-4326-a092-f80bbfcc3c08 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:23.543614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b9b89-2cc3-48bb-becb-6b27f85b60e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:23.544177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b9b89-2cc3-48bb-becb-6b27f85b60e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:23.544554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b9b89-2cc3-48bb-becb-6b27f85b60e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:23.544828 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b9b89-2cc3-48bb-becb-6b27f85b60e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.545500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b9b89-2cc3-48bb-becb-6b27f85b60e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:23.547604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b9b89-2cc3-48bb-becb-6b27f85b60e9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:23.563517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1272/2539] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:23.565181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1053328-c823-4e81-aed0-9461c128b957 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:23.578311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1053328-c823-4e81-aed0-9461c128b957 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:23.583863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b506b27-1236-44de-8ea1-04b083b7015a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:23.584531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b506b27-1236-44de-8ea1-04b083b7015a admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:23.584811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b506b27-1236-44de-8ea1-04b083b7015a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:23.585092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b506b27-1236-44de-8ea1-04b083b7015a admin admin] PATH_INFO: `/v3/projects/3c88c8ebf6844bde97a879172754cc27/users/b06bd166192b42f6a8b5ffb1af2f29b3/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.600085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b506b27-1236-44de-8ea1-04b083b7015a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c88c8ebf6844bde97a879172754cc27, user_id=b06bd166192b42f6a8b5ffb1af2f29b3, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:23.602960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b506b27-1236-44de-8ea1-04b083b7015a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:23.709718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1268/2540] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:23 2022] PUT /identity/v3/projects/3c88c8ebf6844bde97a879172754cc27/users/b06bd166192b42f6a8b5ffb1af2f29b3/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:23.711571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934a399-5bd8-4bdf-aebe-f870d79ff59e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:23.711860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934a399-5bd8-4bdf-aebe-f870d79ff59e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:23.712164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934a399-5bd8-4bdf-aebe-f870d79ff59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.741450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3934a399-5bd8-4bdf-aebe-f870d79ff59e None None] MFA Rules not processed for user `b06bd166192b42f6a8b5ffb1af2f29b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:23.743392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3934a399-5bd8-4bdf-aebe-f870d79ff59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:23.766214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:23.818674 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1273/2541] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:23.820261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b506b27-1236-44de-8ea1-04b083b7015a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:23.842786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:23.916852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b506b27-1236-44de-8ea1-04b083b7015a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:23.924871 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd250d-7f0f-410e-9fbd-5e22c1620190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:23.926007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd250d-7f0f-410e-9fbd-5e22c1620190 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:23.926500 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd250d-7f0f-410e-9fbd-5e22c1620190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:23.927043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd250d-7f0f-410e-9fbd-5e22c1620190 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:23.930322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd250d-7f0f-410e-9fbd-5e22c1620190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:23.933269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd250d-7f0f-410e-9fbd-5e22c1620190 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:24.075901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1269/2542] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:23 2022] POST /identity/v3/projects => generated 375 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:24.084225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934a399-5bd8-4bdf-aebe-f870d79ff59e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:24.098768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934a399-5bd8-4bdf-aebe-f870d79ff59e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:24.172363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878df58-2b5a-4e8c-98d2-68ef93736b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:24.172992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878df58-2b5a-4e8c-98d2-68ef93736b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:24.173303 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878df58-2b5a-4e8c-98d2-68ef93736b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:24.173706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878df58-2b5a-4e8c-98d2-68ef93736b60 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:24.174211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878df58-2b5a-4e8c-98d2-68ef93736b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:24.176219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878df58-2b5a-4e8c-98d2-68ef93736b60 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:24.333735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1274/2543] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:24 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:24.338302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd250d-7f0f-410e-9fbd-5e22c1620190 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:24.348723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd250d-7f0f-410e-9fbd-5e22c1620190 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:24.358887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00691465-000c-49db-8ad4-f28fd11ceebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:24.358887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00691465-000c-49db-8ad4-f28fd11ceebe admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:24.358887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00691465-000c-49db-8ad4-f28fd11ceebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:24.358887 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00691465-000c-49db-8ad4-f28fd11ceebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:24.359376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00691465-000c-49db-8ad4-f28fd11ceebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:24.359376 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00691465-000c-49db-8ad4-f28fd11ceebe admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:24.370304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1270/2544] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:24.374583 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878df58-2b5a-4e8c-98d2-68ef93736b60 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:24.385670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878df58-2b5a-4e8c-98d2-68ef93736b60 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:24.391295 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276ec7c-91da-4769-a834-d8c798e1350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:24.392187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276ec7c-91da-4769-a834-d8c798e1350c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:24.392687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276ec7c-91da-4769-a834-d8c798e1350c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:24.393159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276ec7c-91da-4769-a834-d8c798e1350c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:24.394224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276ec7c-91da-4769-a834-d8c798e1350c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:24.395933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276ec7c-91da-4769-a834-d8c798e1350c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:24.408230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1275/2545] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:24.412673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00691465-000c-49db-8ad4-f28fd11ceebe admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:24.426724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00691465-000c-49db-8ad4-f28fd11ceebe admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:24.431595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e6f32-bc27-4c77-ad34-500027c18e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:24.432247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e6f32-bc27-4c77-ad34-500027c18e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:24.432526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e6f32-bc27-4c77-ad34-500027c18e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:24.432794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e6f32-bc27-4c77-ad34-500027c18e6a admin admin] PATH_INFO: `/v3/projects/08114aee4deb4bb4bcd6ce07fc7f39ef/users/9a8a27b8bb0942febc9b1dccee5d5af8/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:24.448776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e6f32-bc27-4c77-ad34-500027c18e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=08114aee4deb4bb4bcd6ce07fc7f39ef, user_id=9a8a27b8bb0942febc9b1dccee5d5af8, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:24.451603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e6f32-bc27-4c77-ad34-500027c18e6a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:24.581601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1271/2546] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:24 2022] PUT /identity/v3/projects/08114aee4deb4bb4bcd6ce07fc7f39ef/users/9a8a27b8bb0942febc9b1dccee5d5af8/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:24.595912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a48766-5876-47ec-9c95-c318f66dcf97 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:24.596201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a48766-5876-47ec-9c95-c318f66dcf97 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:24.596476 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a48766-5876-47ec-9c95-c318f66dcf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:24.626646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e5a48766-5876-47ec-9c95-c318f66dcf97 None None] MFA Rules not processed for user `9a8a27b8bb0942febc9b1dccee5d5af8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:24.630797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e5a48766-5876-47ec-9c95-c318f66dcf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:24.700654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1276/2547] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:24.705202 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e6f32-bc27-4c77-ad34-500027c18e6a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:24.780110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e6f32-bc27-4c77-ad34-500027c18e6a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:24.785368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af37e5f-c9a0-430d-a13d-61140d1f1ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:24.786963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af37e5f-c9a0-430d-a13d-61140d1f1ffd admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:24.791395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af37e5f-c9a0-430d-a13d-61140d1f1ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:24.791395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af37e5f-c9a0-430d-a13d-61140d1f1ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:24.791395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af37e5f-c9a0-430d-a13d-61140d1f1ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:24.791395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af37e5f-c9a0-430d-a13d-61140d1f1ffd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:24.906287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1272/2548] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:24 2022] POST /identity/v3/projects => generated 377 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:24.910408 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a48766-5876-47ec-9c95-c318f66dcf97 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:24.921171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a48766-5876-47ec-9c95-c318f66dcf97 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:24.954027 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:24.983088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d6a20-af92-4ac1-bd9f-381956a5bed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:24.983782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d6a20-af92-4ac1-bd9f-381956a5bed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:24.984011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d6a20-af92-4ac1-bd9f-381956a5bed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:24.984290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d6a20-af92-4ac1-bd9f-381956a5bed1 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:24.984977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d6a20-af92-4ac1-bd9f-381956a5bed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:24.987067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d6a20-af92-4ac1-bd9f-381956a5bed1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:25.086182 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af37e5f-c9a0-430d-a13d-61140d1f1ffd admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:25.119630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1277/2549] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:24 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:25.126074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af37e5f-c9a0-430d-a13d-61140d1f1ffd admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:25.126074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4df731-c9f6-469f-adef-e1dce6f138a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:25.126074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df731-c9f6-469f-adef-e1dce6f138a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.126074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df731-c9f6-469f-adef-e1dce6f138a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.126962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df731-c9f6-469f-adef-e1dce6f138a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.126962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4df731-c9f6-469f-adef-e1dce6f138a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:25.126962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4df731-c9f6-469f-adef-e1dce6f138a4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:25.126962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1273/2550] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:25.136113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d6a20-af92-4ac1-bd9f-381956a5bed1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:25.150686 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d6a20-af92-4ac1-bd9f-381956a5bed1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:25.156564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cb85d-9660-4057-bd3c-4d44ade5a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:25.157442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cb85d-9660-4057-bd3c-4d44ade5a309 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.157940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cb85d-9660-4057-bd3c-4d44ade5a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.158457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cb85d-9660-4057-bd3c-4d44ade5a309 admin admin] PATH_INFO: `/v3/projects/8d10a058717d4e51b6bb1864b8fd894d/users/0c47c3011b0342dca62137a486ab0d8c/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.212428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cb85d-9660-4057-bd3c-4d44ade5a309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d10a058717d4e51b6bb1864b8fd894d, user_id=0c47c3011b0342dca62137a486ab0d8c, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:25.235563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cb85d-9660-4057-bd3c-4d44ade5a309 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:25.619149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1278/2551] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:25 2022] PUT /identity/v3/projects/8d10a058717d4e51b6bb1864b8fd894d/users/0c47c3011b0342dca62137a486ab0d8c/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:25.621673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4df731-c9f6-469f-adef-e1dce6f138a4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:25.632777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4df731-c9f6-469f-adef-e1dce6f138a4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:25.677310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:25.698249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f5171-a3a7-4675-93dd-de83976a8e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:25.698863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f5171-a3a7-4675-93dd-de83976a8e52 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.699145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f5171-a3a7-4675-93dd-de83976a8e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.699479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f5171-a3a7-4675-93dd-de83976a8e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.700132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f5171-a3a7-4675-93dd-de83976a8e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:25.701734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f5171-a3a7-4675-93dd-de83976a8e52 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:25.716243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1274/2552] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:25.717335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cb85d-9660-4057-bd3c-4d44ade5a309 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:25.729766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cb85d-9660-4057-bd3c-4d44ade5a309 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:25.734919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7b274-307d-4f7c-a5a4-0c8d3dd14c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:25.736003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7b274-307d-4f7c-a5a4-0c8d3dd14c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.736003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7b274-307d-4f7c-a5a4-0c8d3dd14c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.736450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7b274-307d-4f7c-a5a4-0c8d3dd14c9a admin admin] PATH_INFO: `/v3/projects/8d10a058717d4e51b6bb1864b8fd894d/users/0c47c3011b0342dca62137a486ab0d8c/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.739073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db7b274-307d-4f7c-a5a4-0c8d3dd14c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d10a058717d4e51b6bb1864b8fd894d, user_id=0c47c3011b0342dca62137a486ab0d8c, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:25.745719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db7b274-307d-4f7c-a5a4-0c8d3dd14c9a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:25.762480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1279/2553] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:25 2022] PUT /identity/v3/projects/8d10a058717d4e51b6bb1864b8fd894d/users/0c47c3011b0342dca62137a486ab0d8c/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:25.772776 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e161bb6-a37e-4587-a844-40ff25af5f74 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.773160 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e161bb6-a37e-4587-a844-40ff25af5f74 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.773378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e161bb6-a37e-4587-a844-40ff25af5f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.801924 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8e161bb6-a37e-4587-a844-40ff25af5f74 None None] MFA Rules not processed for user `0c47c3011b0342dca62137a486ab0d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:25.804058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8e161bb6-a37e-4587-a844-40ff25af5f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:25.881780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1275/2554] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:25.893672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7b274-307d-4f7c-a5a4-0c8d3dd14c9a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:25.895379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9db7b274-307d-4f7c-a5a4-0c8d3dd14c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:25.969118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7b274-307d-4f7c-a5a4-0c8d3dd14c9a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:25.969118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714acc5-cae0-43c8-9c65-ecebd86c967a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:25.969118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714acc5-cae0-43c8-9c65-ecebd86c967a service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.969118 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714acc5-cae0-43c8-9c65-ecebd86c967a service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.969540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714acc5-cae0-43c8-9c65-ecebd86c967a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.046004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714acc5-cae0-43c8-9c65-ecebd86c967a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.047825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714acc5-cae0-43c8-9c65-ecebd86c967a service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.059863 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1280/2555] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:29.358701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e161bb6-a37e-4587-a844-40ff25af5f74 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:29.360309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8e161bb6-a37e-4587-a844-40ff25af5f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:29.373556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e161bb6-a37e-4587-a844-40ff25af5f74 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:29.378952 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bbea9-6f9f-44b0-911e-cf78fcb9e7a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:29.379566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bbea9-6f9f-44b0-911e-cf78fcb9e7a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:29.379895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bbea9-6f9f-44b0-911e-cf78fcb9e7a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:29.380172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bbea9-6f9f-44b0-911e-cf78fcb9e7a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:29.441296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bbea9-6f9f-44b0-911e-cf78fcb9e7a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:29.442780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bbea9-6f9f-44b0-911e-cf78fcb9e7a9 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:29.452145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1276/2556] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:31.208773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714acc5-cae0-43c8-9c65-ecebd86c967a service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:31.218186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714acc5-cae0-43c8-9c65-ecebd86c967a service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:31.221960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b5a09-2588-49dc-9058-aa0e9f9c3f4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:31.222628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5a09-2588-49dc-9058-aa0e9f9c3f4c service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:31.223013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5a09-2588-49dc-9058-aa0e9f9c3f4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:31.223361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5a09-2588-49dc-9058-aa0e9f9c3f4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:31.233563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5a09-2588-49dc-9058-aa0e9f9c3f4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:31.235088 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5a09-2588-49dc-9058-aa0e9f9c3f4c service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:31.245020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1281/2557] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:37.184353 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bbea9-6f9f-44b0-911e-cf78fcb9e7a9 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:37.246215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bbea9-6f9f-44b0-911e-cf78fcb9e7a9 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:37.250755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:37.251382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:37.251662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:37.251935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] PATH_INFO: `/v3/users/b06bd166192b42f6a8b5ffb1af2f29b3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:37.253050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b06bd166192b42f6a8b5ffb1af2f29b3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:37.255036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:37.256543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:37.449020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b06bd166192b42f6a8b5ffb1af2f29b3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:37.449675 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b06bd166192b42f6a8b5ffb1af2f29b3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:37.450302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b06bd166192b42f6a8b5ffb1af2f29b3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:37.566722 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b06bd166192b42f6a8b5ffb1af2f29b3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:37.575561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b06bd166192b42f6a8b5ffb1af2f29b3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:37.595663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1277/2558] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:37 2022] DELETE /identity/v3/users/b06bd166192b42f6a8b5ffb1af2f29b3 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:37.600299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb45563-60da-42b7-959e-1260272afee7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:37.600776 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb45563-60da-42b7-959e-1260272afee7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:37.600864 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb45563-60da-42b7-959e-1260272afee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:37.621418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9cb45563-60da-42b7-959e-1260272afee7 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:37.622745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9cb45563-60da-42b7-959e-1260272afee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:37.643510 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:37.685727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1282/2559] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:37.695400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:37.696898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:37.753967 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d7a3c-f621-4503-bc68-78d0fcd420fc admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:37.757548 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48943c5c-3973-4286-86f6-dc97ad23ea13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:37.758175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48943c5c-3973-4286-86f6-dc97ad23ea13 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:37.758449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48943c5c-3973-4286-86f6-dc97ad23ea13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:37.758715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48943c5c-3973-4286-86f6-dc97ad23ea13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:37.768463 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48943c5c-3973-4286-86f6-dc97ad23ea13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:37.769895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48943c5c-3973-4286-86f6-dc97ad23ea13 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:37.779739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1278/2560] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:38.300821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb45563-60da-42b7-959e-1260272afee7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:38.305499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9cb45563-60da-42b7-959e-1260272afee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:38.314513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb45563-60da-42b7-959e-1260272afee7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:38.319090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:38.319732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:38.319996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:38.320269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] PATH_INFO: `/v3/projects/3c88c8ebf6844bde97a879172754cc27` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:38.325975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c88c8ebf6844bde97a879172754cc27)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:38.325975 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:38.459682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c88c8ebf6844bde97a879172754cc27', 'request_id': 'req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:38.465152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c88c8ebf6844bde97a879172754cc27', 'request_id': 'req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:38.592111 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c88c8ebf6844bde97a879172754cc27', 'request_id': 'req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:38.603926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] The token cache is being invalidate because project 3c88c8ebf6844bde97a879172754cc27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:38.604594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:38.604934 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:38.606564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1283/2561] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:38 2022] DELETE /identity/v3/projects/3c88c8ebf6844bde97a879172754cc27 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:38.610601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48943c5c-3973-4286-86f6-dc97ad23ea13 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:38.620879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48943c5c-3973-4286-86f6-dc97ad23ea13 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:38.680770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:38.681684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:38.682122 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:38.682395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] PATH_INFO: `/v3/users/9a8a27b8bb0942febc9b1dccee5d5af8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:38.683646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8a27b8bb0942febc9b1dccee5d5af8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:38.686349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:38.710382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:38.957359 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8a27b8bb0942febc9b1dccee5d5af8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:38.958045 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8a27b8bb0942febc9b1dccee5d5af8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:38.958679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8a27b8bb0942febc9b1dccee5d5af8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:39.072200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8a27b8bb0942febc9b1dccee5d5af8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:39.079861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8a27b8bb0942febc9b1dccee5d5af8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:39.097812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1279/2562] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:38 2022] DELETE /identity/v3/users/9a8a27b8bb0942febc9b1dccee5d5af8 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:39.560271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:39.561725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:39.577566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fe1d6-6787-48ee-ba8b-71e4f98ab763 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:39.635681 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:39.636358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:39.636589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:39.636872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] PATH_INFO: `/v3/projects/08114aee4deb4bb4bcd6ce07fc7f39ef` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:39.638049 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] RBAC: Authorizing `identity:delete_project(project_id=08114aee4deb4bb4bcd6ce07fc7f39ef)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:39.639989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:39.767383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08114aee4deb4bb4bcd6ce07fc7f39ef', 'request_id': 'req-aac5ee4e-ff39-4517-9f40-6d90b693b81a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:39.768570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:39.773504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08114aee4deb4bb4bcd6ce07fc7f39ef', 'request_id': 'req-aac5ee4e-ff39-4517-9f40-6d90b693b81a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:39.900508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08114aee4deb4bb4bcd6ce07fc7f39ef', 'request_id': 'req-aac5ee4e-ff39-4517-9f40-6d90b693b81a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:39.912190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] The token cache is being invalidate because project 08114aee4deb4bb4bcd6ce07fc7f39ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:39.912482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:39.913117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:39.914716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1284/2563] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:39 2022] DELETE /identity/v3/projects/08114aee4deb4bb4bcd6ce07fc7f39ef => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:39.918842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:39.920288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:39.933671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbf188-4402-4bb0-aaa0-02a59c7de6a9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:39.990694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:39.991320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:39.991601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:39.991877 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] PATH_INFO: `/v3/users/0c47c3011b0342dca62137a486ab0d8c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:39.993534 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c47c3011b0342dca62137a486ab0d8c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:39.995474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:40.294843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c47c3011b0342dca62137a486ab0d8c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:40.295499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c47c3011b0342dca62137a486ab0d8c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:40.296099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c47c3011b0342dca62137a486ab0d8c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:40.401125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c47c3011b0342dca62137a486ab0d8c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:40.409036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d18a7202-03ae-426c-927a-39e393e6e429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c47c3011b0342dca62137a486ab0d8c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:40.427062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1280/2564] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:39 2022] DELETE /identity/v3/users/0c47c3011b0342dca62137a486ab0d8c => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:40.938866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:40.938866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:40.953715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ee4e-ff39-4517-9f40-6d90b693b81a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:41.013537 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:41.014536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.014961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.015243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] PATH_INFO: `/v3/projects/8d10a058717d4e51b6bb1864b8fd894d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.016463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d10a058717d4e51b6bb1864b8fd894d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:41.018544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:41.164147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d10a058717d4e51b6bb1864b8fd894d', 'request_id': 'req-3b88c67b-7ba0-440f-a777-555beddec6a9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:41.169788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d10a058717d4e51b6bb1864b8fd894d', 'request_id': 'req-3b88c67b-7ba0-440f-a777-555beddec6a9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:41.266283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d10a058717d4e51b6bb1864b8fd894d', 'request_id': 'req-3b88c67b-7ba0-440f-a777-555beddec6a9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:41.278618 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] The token cache is being invalidate because project 8d10a058717d4e51b6bb1864b8fd894d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:41.278926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:41.279617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3b88c67b-7ba0-440f-a777-555beddec6a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:41.281113 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1285/2565] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:40 2022] DELETE /identity/v3/projects/8d10a058717d4e51b6bb1864b8fd894d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:41.286853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1f7b-53bf-48db-8d74-00b1ff803a38 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.287342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1f7b-53bf-48db-8d74-00b1ff803a38 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.287453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1f7b-53bf-48db-8d74-00b1ff803a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.306499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e69f1f7b-53bf-48db-8d74-00b1ff803a38 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:41.307682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e69f1f7b-53bf-48db-8d74-00b1ff803a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:41.320203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:41.366456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1281/2566] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:41.375123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273d9b7-1b4d-496c-9a3d-72d92d7b9674 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.375482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273d9b7-1b4d-496c-9a3d-72d92d7b9674 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.375767 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273d9b7-1b4d-496c-9a3d-72d92d7b9674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.396203 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1273d9b7-1b4d-496c-9a3d-72d92d7b9674 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:41.397472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1273d9b7-1b4d-496c-9a3d-72d92d7b9674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:41.405666 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1286/2567] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:41.415193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f1f7b-53bf-48db-8d74-00b1ff803a38 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:41.425235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f1f7b-53bf-48db-8d74-00b1ff803a38 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:41.429936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b885f43-193a-43db-877a-f446efc56f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:41.431078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b885f43-193a-43db-877a-f446efc56f03 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.431356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b885f43-193a-43db-877a-f446efc56f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.431626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b885f43-193a-43db-877a-f446efc56f03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.432276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b885f43-193a-43db-877a-f446efc56f03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:41.432578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b885f43-193a-43db-877a-f446efc56f03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:41.434201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b885f43-193a-43db-877a-f446efc56f03 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:41.447511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1282/2568] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:36:41.455493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273d9b7-1b4d-496c-9a3d-72d92d7b9674 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:41.466160 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273d9b7-1b4d-496c-9a3d-72d92d7b9674 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:41.471213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92331f52-333d-4bca-b727-1712b7dd5c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:41.471897 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92331f52-333d-4bca-b727-1712b7dd5c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.472185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92331f52-333d-4bca-b727-1712b7dd5c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.472453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92331f52-333d-4bca-b727-1712b7dd5c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.473099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92331f52-333d-4bca-b727-1712b7dd5c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:41.475050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92331f52-333d-4bca-b727-1712b7dd5c06 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:41.604416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1287/2569] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:41 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:41.608386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b885f43-193a-43db-877a-f446efc56f03 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:41.631484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b885f43-193a-43db-877a-f446efc56f03 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:41.679630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13e45e-c6cb-4213-9640-c4fb42d3a2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:41.680770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13e45e-c6cb-4213-9640-c4fb42d3a2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.681049 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13e45e-c6cb-4213-9640-c4fb42d3a2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.688559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13e45e-c6cb-4213-9640-c4fb42d3a2d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.688559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13e45e-c6cb-4213-9640-c4fb42d3a2d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:41.688559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13e45e-c6cb-4213-9640-c4fb42d3a2d2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:41.811431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1283/2570] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:41 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:41.812778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92331f52-333d-4bca-b727-1712b7dd5c06 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:41.825483 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92331f52-333d-4bca-b727-1712b7dd5c06 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:41.830299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ecd15-d5b6-4f7b-a8ef-bdeb233f761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:41.831003 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ecd15-d5b6-4f7b-a8ef-bdeb233f761d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.831356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ecd15-d5b6-4f7b-a8ef-bdeb233f761d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.831702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ecd15-d5b6-4f7b-a8ef-bdeb233f761d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.832434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ecd15-d5b6-4f7b-a8ef-bdeb233f761d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:41.834170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ecd15-d5b6-4f7b-a8ef-bdeb233f761d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:41.849191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13e45e-c6cb-4213-9640-c4fb42d3a2d2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:41.850217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1288/2571] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:41.869591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13e45e-c6cb-4213-9640-c4fb42d3a2d2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:41.874668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75673f3a-dfaa-4b63-90c5-e5e53d100c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:41.876235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75673f3a-dfaa-4b63-90c5-e5e53d100c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.876517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75673f3a-dfaa-4b63-90c5-e5e53d100c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.876777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75673f3a-dfaa-4b63-90c5-e5e53d100c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.877641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75673f3a-dfaa-4b63-90c5-e5e53d100c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:41.879278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75673f3a-dfaa-4b63-90c5-e5e53d100c64 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:41.892698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1284/2572] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:36:41.896766 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ecd15-d5b6-4f7b-a8ef-bdeb233f761d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:41.907748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ecd15-d5b6-4f7b-a8ef-bdeb233f761d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:41.913278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fcc00-7426-4e82-8421-5cfeb754059d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:41.914168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fcc00-7426-4e82-8421-5cfeb754059d admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.915099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fcc00-7426-4e82-8421-5cfeb754059d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.917901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fcc00-7426-4e82-8421-5cfeb754059d admin admin] PATH_INFO: `/v3/projects/51b8f02919e448daa31537f626dfaaba/users/baf34d8c84d44acd93c0a4eeb8c3c1ec/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.933199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fcc00-7426-4e82-8421-5cfeb754059d admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b8f02919e448daa31537f626dfaaba, user_id=baf34d8c84d44acd93c0a4eeb8c3c1ec, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:41.940407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fcc00-7426-4e82-8421-5cfeb754059d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:42.080224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1289/2573] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:41 2022] PUT /identity/v3/projects/51b8f02919e448daa31537f626dfaaba/users/baf34d8c84d44acd93c0a4eeb8c3c1ec/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:42.099528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94ded-a30d-4573-b0e2-f536dff93444 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:42.099816 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94ded-a30d-4573-b0e2-f536dff93444 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:42.100116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94ded-a30d-4573-b0e2-f536dff93444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:42.142396 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-78a94ded-a30d-4573-b0e2-f536dff93444 None None] MFA Rules not processed for user `baf34d8c84d44acd93c0a4eeb8c3c1ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:42.143618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-78a94ded-a30d-4573-b0e2-f536dff93444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:42.212744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1285/2574] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:42.223542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fcc00-7426-4e82-8421-5cfeb754059d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:42.225257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-009fcc00-7426-4e82-8421-5cfeb754059d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:42.233517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:42.291141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fcc00-7426-4e82-8421-5cfeb754059d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:42.295527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e0b9-ab51-4d6c-857a-b7773d74ef3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:42.296178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e0b9-ab51-4d6c-857a-b7773d74ef3c service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:42.296605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e0b9-ab51-4d6c-857a-b7773d74ef3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:42.296879 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e0b9-ab51-4d6c-857a-b7773d74ef3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:42.305499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417e0b9-ab51-4d6c-857a-b7773d74ef3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:42.307155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417e0b9-ab51-4d6c-857a-b7773d74ef3c service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:42.316067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1290/2575] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:47.043595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bbb1b-0097-4d09-8def-c7bc640b1ed0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:47.044075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bbb1b-0097-4d09-8def-c7bc640b1ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:47.044190 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bbb1b-0097-4d09-8def-c7bc640b1ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:47.064447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a08bbb1b-0097-4d09-8def-c7bc640b1ed0 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:47.065687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a08bbb1b-0097-4d09-8def-c7bc640b1ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:47.075762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:47.135003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1286/2576] 10.222.0.83 () {58 vars in 967 bytes} [Wed Aug 24 11:36:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:47.147182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e0b9-ab51-4d6c-857a-b7773d74ef3c service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:47.148702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6417e0b9-ab51-4d6c-857a-b7773d74ef3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:47.218768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e0b9-ab51-4d6c-857a-b7773d74ef3c service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:47.223777 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b91f6-90ff-4ba5-8053-994cb7c46ed3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:47.224385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b91f6-90ff-4ba5-8053-994cb7c46ed3 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:47.224692 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b91f6-90ff-4ba5-8053-994cb7c46ed3 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:47.225004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b91f6-90ff-4ba5-8053-994cb7c46ed3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:47.233090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b91f6-90ff-4ba5-8053-994cb7c46ed3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:47.234746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b91f6-90ff-4ba5-8053-994cb7c46ed3 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:47.242800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1291/2577] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:36:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:36:48.858820 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bbb1b-0097-4d09-8def-c7bc640b1ed0 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:48.860695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a08bbb1b-0097-4d09-8def-c7bc640b1ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:49.003052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bbb1b-0097-4d09-8def-c7bc640b1ed0 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:49.010938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b75b3c-6887-4504-a8fe-682368349fb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:49.011701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b75b3c-6887-4504-a8fe-682368349fb0 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:49.012050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b75b3c-6887-4504-a8fe-682368349fb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:49.012397 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b75b3c-6887-4504-a8fe-682368349fb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:49.016825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e0b75b3c-6887-4504-a8fe-682368349fb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:49.030816 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b75b3c-6887-4504-a8fe-682368349fb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:49.032617 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b75b3c-6887-4504-a8fe-682368349fb0 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:49.046040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1287/2578] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:56.264378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b91f6-90ff-4ba5-8053-994cb7c46ed3 service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:56.265891 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7f0b91f6-90ff-4ba5-8053-994cb7c46ed3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:56.331923 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b91f6-90ff-4ba5-8053-994cb7c46ed3 service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:56.335558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de5bda-7b6d-47a8-9a12-b04cce2b1553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:56.337211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de5bda-7b6d-47a8-9a12-b04cce2b1553 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:56.337211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de5bda-7b6d-47a8-9a12-b04cce2b1553 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:56.337418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de5bda-7b6d-47a8-9a12-b04cce2b1553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:56.339029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-46de5bda-7b6d-47a8-9a12-b04cce2b1553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:56.390471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:56.405025 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de5bda-7b6d-47a8-9a12-b04cce2b1553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:56.406580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de5bda-7b6d-47a8-9a12-b04cce2b1553 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:56.416473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1292/2579] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:58.159946 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b75b3c-6887-4504-a8fe-682368349fb0 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:58.172115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b75b3c-6887-4504-a8fe-682368349fb0 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:58.179184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:58.190123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:58.190123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:58.190123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] PATH_INFO: `/v3/users/baf34d8c84d44acd93c0a4eeb8c3c1ec` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:58.190123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf34d8c84d44acd93c0a4eeb8c3c1ec)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:58.190123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:58.366298 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf34d8c84d44acd93c0a4eeb8c3c1ec'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:58.367579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf34d8c84d44acd93c0a4eeb8c3c1ec'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:58.368253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf34d8c84d44acd93c0a4eeb8c3c1ec'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:58.510582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf34d8c84d44acd93c0a4eeb8c3c1ec'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:58.526029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf34d8c84d44acd93c0a4eeb8c3c1ec'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:58.550204 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1288/2580] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:58 2022] DELETE /identity/v3/users/baf34d8c84d44acd93c0a4eeb8c3c1ec => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:58.555195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d682ee-bb04-4c97-bbb7-54a09ad89699 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:58.555915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d682ee-bb04-4c97-bbb7-54a09ad89699 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:58.555915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d682ee-bb04-4c97-bbb7-54a09ad89699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:58.575654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-09d682ee-bb04-4c97-bbb7-54a09ad89699 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:58.576898 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-09d682ee-bb04-4c97-bbb7-54a09ad89699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:58.650059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1293/2581] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:58.656720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:58.658999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:58.685192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:58.724133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86affc-a314-430f-a17d-fe1d92e26961 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:58.729383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e68d63-7186-4ac6-ba72-ac8d812bc914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:58.730027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e68d63-7186-4ac6-ba72-ac8d812bc914 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:58.734315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e68d63-7186-4ac6-ba72-ac8d812bc914 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:58.734315 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e68d63-7186-4ac6-ba72-ac8d812bc914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:58.740165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e68d63-7186-4ac6-ba72-ac8d812bc914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:58.741712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e68d63-7186-4ac6-ba72-ac8d812bc914 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:58.752338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1289/2582] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:59.288061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d682ee-bb04-4c97-bbb7-54a09ad89699 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:59.289593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-09d682ee-bb04-4c97-bbb7-54a09ad89699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:59.302667 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d682ee-bb04-4c97-bbb7-54a09ad89699 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:59.307959 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:59.308457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:59.308731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:59.309004 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] PATH_INFO: `/v3/projects/51b8f02919e448daa31537f626dfaaba` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:59.310272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b8f02919e448daa31537f626dfaaba)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:59.312401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:59.448143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b8f02919e448daa31537f626dfaaba', 'request_id': 'req-f62a6748-83ec-4c3b-9bba-9dfce61085cc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:59.461058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b8f02919e448daa31537f626dfaaba', 'request_id': 'req-f62a6748-83ec-4c3b-9bba-9dfce61085cc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:59.585330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b8f02919e448daa31537f626dfaaba', 'request_id': 'req-f62a6748-83ec-4c3b-9bba-9dfce61085cc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:59.597333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] The token cache is being invalidate because project 51b8f02919e448daa31537f626dfaaba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:59.597821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:59.598353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f62a6748-83ec-4c3b-9bba-9dfce61085cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:59.600029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1294/2583] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:59 2022] DELETE /identity/v3/projects/51b8f02919e448daa31537f626dfaaba => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:59.604884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085509b-cce7-4fc5-9091-c1a7e47f7446 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:59.605338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085509b-cce7-4fc5-9091-c1a7e47f7446 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:59.605451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085509b-cce7-4fc5-9091-c1a7e47f7446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:59.637821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a085509b-cce7-4fc5-9091-c1a7e47f7446 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:59.637821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a085509b-cce7-4fc5-9091-c1a7e47f7446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:59.713540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1290/2584] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:59.715668 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592d829-6585-4fad-978a-35c3821ea1eb None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:59.716127 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592d829-6585-4fad-978a-35c3821ea1eb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:59.716582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592d829-6585-4fad-978a-35c3821ea1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:59.747533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e592d829-6585-4fad-978a-35c3821ea1eb None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:59.749450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e592d829-6585-4fad-978a-35c3821ea1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:59.770093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1295/2585] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:59.771451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085509b-cce7-4fc5-9091-c1a7e47f7446 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:59.783021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085509b-cce7-4fc5-9091-c1a7e47f7446 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:59.788157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f152488-15bc-4d61-b402-9af19618659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:59.789318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f152488-15bc-4d61-b402-9af19618659e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:59.789601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f152488-15bc-4d61-b402-9af19618659e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:59.789897 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f152488-15bc-4d61-b402-9af19618659e admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:59.790601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f152488-15bc-4d61-b402-9af19618659e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:59.790909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f152488-15bc-4d61-b402-9af19618659e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:59.792530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f152488-15bc-4d61-b402-9af19618659e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:59.809529 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1291/2586] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:36:59.813814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592d829-6585-4fad-978a-35c3821ea1eb None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:59.827920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592d829-6585-4fad-978a-35c3821ea1eb None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:59.832763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae85b23-372c-4ab2-b24c-0927ca506bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:59.833436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85b23-372c-4ab2-b24c-0927ca506bdb admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:59.835169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85b23-372c-4ab2-b24c-0927ca506bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:59.835169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85b23-372c-4ab2-b24c-0927ca506bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:59.835169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae85b23-372c-4ab2-b24c-0927ca506bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:59.837283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae85b23-372c-4ab2-b24c-0927ca506bdb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:59.953677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1296/2587] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:59 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:59.958829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f152488-15bc-4d61-b402-9af19618659e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:59.976855 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f152488-15bc-4d61-b402-9af19618659e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:00.036417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a5854-8b02-43a5-954d-b7d6ef1b7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:00.037030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a5854-8b02-43a5-954d-b7d6ef1b7c21 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:00.037302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a5854-8b02-43a5-954d-b7d6ef1b7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:00.037622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a5854-8b02-43a5-954d-b7d6ef1b7c21 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:00.038292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a5854-8b02-43a5-954d-b7d6ef1b7c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:00.040821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a5854-8b02-43a5-954d-b7d6ef1b7c21 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:00.278082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1292/2588] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:59 2022] POST /identity/v3/users => generated 288 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:37:00.281284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae85b23-372c-4ab2-b24c-0927ca506bdb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:00.297600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae85b23-372c-4ab2-b24c-0927ca506bdb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:00.303613 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b059e6-a848-4870-af53-323017116a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:00.304392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b059e6-a848-4870-af53-323017116a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:00.304729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b059e6-a848-4870-af53-323017116a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:00.305073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b059e6-a848-4870-af53-323017116a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:00.314419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b059e6-a848-4870-af53-323017116a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:00.314419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b059e6-a848-4870-af53-323017116a6a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:00.322732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1297/2589] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:37:00.334287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a5854-8b02-43a5-954d-b7d6ef1b7c21 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:00.346966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a5854-8b02-43a5-954d-b7d6ef1b7c21 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:00.352262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4949c1a-a63e-4703-a1c6-98091bbf466f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:00.352860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4949c1a-a63e-4703-a1c6-98091bbf466f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:00.353145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4949c1a-a63e-4703-a1c6-98091bbf466f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:00.353417 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4949c1a-a63e-4703-a1c6-98091bbf466f admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:00.354108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4949c1a-a63e-4703-a1c6-98091bbf466f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:00.355718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4949c1a-a63e-4703-a1c6-98091bbf466f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:00.369775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1293/2590] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:00.371023 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b059e6-a848-4870-af53-323017116a6a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:00.384044 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b059e6-a848-4870-af53-323017116a6a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:00.389125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc5035-1fff-4e3d-ac5a-11dec02405c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:00.389930 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc5035-1fff-4e3d-ac5a-11dec02405c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:00.390217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc5035-1fff-4e3d-ac5a-11dec02405c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:00.390495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc5035-1fff-4e3d-ac5a-11dec02405c1 admin admin] PATH_INFO: `/v3/projects/b4d27b24d8e5430c93890088fb6386db/users/effc45029842477e92552609037f2e35/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:00.406347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc5035-1fff-4e3d-ac5a-11dec02405c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4d27b24d8e5430c93890088fb6386db, user_id=effc45029842477e92552609037f2e35, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:00.409068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc5035-1fff-4e3d-ac5a-11dec02405c1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:00.553095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1298/2591] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:00 2022] PUT /identity/v3/projects/b4d27b24d8e5430c93890088fb6386db/users/effc45029842477e92552609037f2e35/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:37:00.553699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4949c1a-a63e-4703-a1c6-98091bbf466f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:00.555564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a4949c1a-a63e-4703-a1c6-98091bbf466f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:00.571519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4949c1a-a63e-4703-a1c6-98091bbf466f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:00.601308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:00.615778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1352f0-0c1f-4ebf-ba47-7abeb3d8885f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:00.627271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1352f0-0c1f-4ebf-ba47-7abeb3d8885f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:00.627271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1352f0-0c1f-4ebf-ba47-7abeb3d8885f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:00.630375 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:00.636890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:00.637688 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:00.638007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:00.638286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] PATH_INFO: `/v3/users/92712c36310c440c94da6c432c77a895` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:00.639633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92712c36310c440c94da6c432c77a895)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:00.641575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:00.646347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8b1352f0-0c1f-4ebf-ba47-7abeb3d8885f None None] MFA Rules not processed for user `effc45029842477e92552609037f2e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:00.647584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8b1352f0-0c1f-4ebf-ba47-7abeb3d8885f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:00.713030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1299/2592] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:37:00.723708 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1352f0-0c1f-4ebf-ba47-7abeb3d8885f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:00.725053 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8b1352f0-0c1f-4ebf-ba47-7abeb3d8885f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:00.781247 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1352f0-0c1f-4ebf-ba47-7abeb3d8885f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:00.784917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06bc90-9014-479e-bec2-2ded00c7d8cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:00.785552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06bc90-9014-479e-bec2-2ded00c7d8cd service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:00.785835 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06bc90-9014-479e-bec2-2ded00c7d8cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:00.786138 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06bc90-9014-479e-bec2-2ded00c7d8cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:00.794432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06bc90-9014-479e-bec2-2ded00c7d8cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:00.795951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06bc90-9014-479e-bec2-2ded00c7d8cd service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:00.804899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1300/2593] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:37:00.902180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92712c36310c440c94da6c432c77a895'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:00.911268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92712c36310c440c94da6c432c77a895'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:00.911268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92712c36310c440c94da6c432c77a895'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:01.037003 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92712c36310c440c94da6c432c77a895'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:01.044524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a4709d62-a026-49a8-8101-8a9590fce735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92712c36310c440c94da6c432c77a895'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:01.062753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1294/2594] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:00 2022] DELETE /identity/v3/users/92712c36310c440c94da6c432c77a895 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:01.442719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06bc90-9014-479e-bec2-2ded00c7d8cd service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:01.467023 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06bc90-9014-479e-bec2-2ded00c7d8cd service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:01.531177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:01.531808 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:01.532084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:01.532355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] PATH_INFO: `/v3/projects/6e41187605624091a4edc1993c4357e4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:01.533506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e41187605624091a4edc1993c4357e4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:01.535467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:01.743344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e41187605624091a4edc1993c4357e4', 'request_id': 'req-89cef577-dce8-4d33-a509-8450cededebf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:01.749084 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e41187605624091a4edc1993c4357e4', 'request_id': 'req-89cef577-dce8-4d33-a509-8450cededebf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:01.836165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e41187605624091a4edc1993c4357e4', 'request_id': 'req-89cef577-dce8-4d33-a509-8450cededebf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:01.848418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] The token cache is being invalidate because project 6e41187605624091a4edc1993c4357e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:01.848717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:01.849373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-89cef577-dce8-4d33-a509-8450cededebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:01.851107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1301/2595] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:01 2022] DELETE /identity/v3/projects/6e41187605624091a4edc1993c4357e4 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:01.856838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb66aad-ae15-43f0-b514-e8a73f91ac21 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:01.857291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb66aad-ae15-43f0-b514-e8a73f91ac21 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:01.857579 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb66aad-ae15-43f0-b514-e8a73f91ac21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:01.877401 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-eeb66aad-ae15-43f0-b514-e8a73f91ac21 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:01.878663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-eeb66aad-ae15-43f0-b514-e8a73f91ac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:01.939841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1295/2596] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:01.947673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6699e-0140-4e1d-a682-738806f44e58 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:01.948026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6699e-0140-4e1d-a682-738806f44e58 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:01.948367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6699e-0140-4e1d-a682-738806f44e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:01.958796 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:01.971006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b1a6699e-0140-4e1d-a682-738806f44e58 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:01.972234 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b1a6699e-0140-4e1d-a682-738806f44e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:01.980830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1302/2597] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:01.985216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb66aad-ae15-43f0-b514-e8a73f91ac21 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:01.995189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb66aad-ae15-43f0-b514-e8a73f91ac21 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:02.004619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330a87c-2c29-44cb-a13b-bbbe3a9b4b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:02.004619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330a87c-2c29-44cb-a13b-bbbe3a9b4b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:02.004619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330a87c-2c29-44cb-a13b-bbbe3a9b4b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:02.004619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330a87c-2c29-44cb-a13b-bbbe3a9b4b92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:02.005025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330a87c-2c29-44cb-a13b-bbbe3a9b4b92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:37:02.005025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330a87c-2c29-44cb-a13b-bbbe3a9b4b92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:02.005025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330a87c-2c29-44cb-a13b-bbbe3a9b4b92 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:02.016245 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1296/2598] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:37:02.020655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6699e-0140-4e1d-a682-738806f44e58 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:02.030726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6699e-0140-4e1d-a682-738806f44e58 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:02.035501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b43c2-5985-4b97-a1ae-fc7d0b707f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:02.036097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b43c2-5985-4b97-a1ae-fc7d0b707f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:02.036379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b43c2-5985-4b97-a1ae-fc7d0b707f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:02.036648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b43c2-5985-4b97-a1ae-fc7d0b707f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:02.037273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b43c2-5985-4b97-a1ae-fc7d0b707f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:02.039248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b43c2-5985-4b97-a1ae-fc7d0b707f30 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:02.136297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1303/2599] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:02 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:02.140559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330a87c-2c29-44cb-a13b-bbbe3a9b4b92 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:02.151307 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330a87c-2c29-44cb-a13b-bbbe3a9b4b92 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:02.181823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:02.209974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aabac-23c6-4096-890c-e900962df72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:02.210541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aabac-23c6-4096-890c-e900962df72c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:02.211415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aabac-23c6-4096-890c-e900962df72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:02.211695 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aabac-23c6-4096-890c-e900962df72c admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:02.212455 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51aabac-23c6-4096-890c-e900962df72c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:02.214474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51aabac-23c6-4096-890c-e900962df72c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:02.328038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1297/2600] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:02 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:02.331396 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b43c2-5985-4b97-a1ae-fc7d0b707f30 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:02.347067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b43c2-5985-4b97-a1ae-fc7d0b707f30 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:02.355034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bc86c-eea7-4ed1-9cb3-700c29463878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:02.356284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bc86c-eea7-4ed1-9cb3-700c29463878 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:02.356785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bc86c-eea7-4ed1-9cb3-700c29463878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:02.357290 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bc86c-eea7-4ed1-9cb3-700c29463878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:02.358529 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bc86c-eea7-4ed1-9cb3-700c29463878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:02.361121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bc86c-eea7-4ed1-9cb3-700c29463878 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:02.372095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1304/2601] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:02.376430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aabac-23c6-4096-890c-e900962df72c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:02.387735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aabac-23c6-4096-890c-e900962df72c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:02.392355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550d06b-face-4eaf-8fa2-209bb4f8d260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:02.392945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550d06b-face-4eaf-8fa2-209bb4f8d260 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:02.393224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550d06b-face-4eaf-8fa2-209bb4f8d260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:02.393497 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550d06b-face-4eaf-8fa2-209bb4f8d260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:02.394180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550d06b-face-4eaf-8fa2-209bb4f8d260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:02.395765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550d06b-face-4eaf-8fa2-209bb4f8d260 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:02.406484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1298/2602] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:02.410770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bc86c-eea7-4ed1-9cb3-700c29463878 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:02.421117 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bc86c-eea7-4ed1-9cb3-700c29463878 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:02.425813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aebc2-691b-4149-9775-96ef8cde21ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:02.426506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aebc2-691b-4149-9775-96ef8cde21ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:02.426785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aebc2-691b-4149-9775-96ef8cde21ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:02.427063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aebc2-691b-4149-9775-96ef8cde21ad admin admin] PATH_INFO: `/v3/projects/a4d2c15301d44e92b86d7a19347de166/users/77b31c0002ba4ef69c6cb860dbf6dfcc/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:02.442381 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aebc2-691b-4149-9775-96ef8cde21ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d2c15301d44e92b86d7a19347de166, user_id=77b31c0002ba4ef69c6cb860dbf6dfcc, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:02.445633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aebc2-691b-4149-9775-96ef8cde21ad admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:02.544105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1305/2603] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:02 2022] PUT /identity/v3/projects/a4d2c15301d44e92b86d7a19347de166/users/77b31c0002ba4ef69c6cb860dbf6dfcc/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:02.562117 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263c8d-91f9-4929-80fd-38865809c357 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:02.562403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263c8d-91f9-4929-80fd-38865809c357 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:02.564123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263c8d-91f9-4929-80fd-38865809c357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:02.594462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6f263c8d-91f9-4929-80fd-38865809c357 None None] MFA Rules not processed for user `77b31c0002ba4ef69c6cb860dbf6dfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:02.595698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6f263c8d-91f9-4929-80fd-38865809c357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:02.660228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1299/2604] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:02.672344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aebc2-691b-4149-9775-96ef8cde21ad admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:02.674129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-559aebc2-691b-4149-9775-96ef8cde21ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:02.746999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aebc2-691b-4149-9775-96ef8cde21ad admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:02.752956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c26c31-3e09-4a48-934f-0af04d2b3008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:02.758732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c26c31-3e09-4a48-934f-0af04d2b3008 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:02.758732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c26c31-3e09-4a48-934f-0af04d2b3008 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:02.758732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c26c31-3e09-4a48-934f-0af04d2b3008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:02.762861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c26c31-3e09-4a48-934f-0af04d2b3008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:02.764463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c26c31-3e09-4a48-934f-0af04d2b3008 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:02.774519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1306/2605] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:02.989784 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263c8d-91f9-4929-80fd-38865809c357 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:03.054025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263c8d-91f9-4929-80fd-38865809c357 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:03.058471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:03.059100 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:03.059380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:03.059653 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] PATH_INFO: `/v3/users/77b31c0002ba4ef69c6cb860dbf6dfcc` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:03.060786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b31c0002ba4ef69c6cb860dbf6dfcc)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:03.062753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:03.315438 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:03.317719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b31c0002ba4ef69c6cb860dbf6dfcc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:03.318406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b31c0002ba4ef69c6cb860dbf6dfcc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:03.319025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b31c0002ba4ef69c6cb860dbf6dfcc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:03.388484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b31c0002ba4ef69c6cb860dbf6dfcc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:03.395794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b31c0002ba4ef69c6cb860dbf6dfcc'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:03.413665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1300/2606] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:02 2022] DELETE /identity/v3/users/77b31c0002ba4ef69c6cb860dbf6dfcc => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:03.418813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626825e4-cc79-40df-82a1-55ec9e1c82fa None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:03.419069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626825e4-cc79-40df-82a1-55ec9e1c82fa None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:03.419344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626825e4-cc79-40df-82a1-55ec9e1c82fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:03.439284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-626825e4-cc79-40df-82a1-55ec9e1c82fa None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:03.440482 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-626825e4-cc79-40df-82a1-55ec9e1c82fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:03.486814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:03.501579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1307/2607] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:03.511684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:03.513087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:03.571110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f153f-098d-4425-ac72-251bbb69708e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:03.574672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd537f79-8bbd-4c0a-89c7-413d5e2557a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:03.575276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd537f79-8bbd-4c0a-89c7-413d5e2557a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:03.575556 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd537f79-8bbd-4c0a-89c7-413d5e2557a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:03.575829 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd537f79-8bbd-4c0a-89c7-413d5e2557a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:03.585076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd537f79-8bbd-4c0a-89c7-413d5e2557a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:03.586574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd537f79-8bbd-4c0a-89c7-413d5e2557a1 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:03.596295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1301/2608] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:03.900248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e47626-0484-472b-beeb-798bf59f0557 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:03.900778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e47626-0484-472b-beeb-798bf59f0557 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:03.900778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e47626-0484-472b-beeb-798bf59f0557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:03.920625 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-38e47626-0484-472b-beeb-798bf59f0557 None None] MFA Rules not processed for user `effc45029842477e92552609037f2e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:03.921694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-38e47626-0484-472b-beeb-798bf59f0557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:03.977986 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1308/2609] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:04.042663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd537f79-8bbd-4c0a-89c7-413d5e2557a1 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:04.051091 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cd537f79-8bbd-4c0a-89c7-413d5e2557a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:04.067890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd537f79-8bbd-4c0a-89c7-413d5e2557a1 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:04.073105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:04.073377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:04.074223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:04.074844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] PATH_INFO: `/v3/projects/a4d2c15301d44e92b86d7a19347de166` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:04.076197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d2c15301d44e92b86d7a19347de166)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:04.078166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:04.214565 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d2c15301d44e92b86d7a19347de166', 'request_id': 'req-194e612b-0505-478f-b1fd-7e77282444cf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:04.220555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d2c15301d44e92b86d7a19347de166', 'request_id': 'req-194e612b-0505-478f-b1fd-7e77282444cf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:04.227669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d2c15301d44e92b86d7a19347de166', 'request_id': 'req-194e612b-0505-478f-b1fd-7e77282444cf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:04.308968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] The token cache is being invalidate because project a4d2c15301d44e92b86d7a19347de166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:04.309269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:04.309933 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-194e612b-0505-478f-b1fd-7e77282444cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:04.311478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1302/2610] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:04 2022] DELETE /identity/v3/projects/a4d2c15301d44e92b86d7a19347de166 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:04.316311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423e9d9-f7a3-458c-a19b-ce9bf8f93439 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:04.322475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423e9d9-f7a3-458c-a19b-ce9bf8f93439 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:04.322475 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423e9d9-f7a3-458c-a19b-ce9bf8f93439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:04.339164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5423e9d9-f7a3-458c-a19b-ce9bf8f93439 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:04.340386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5423e9d9-f7a3-458c-a19b-ce9bf8f93439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:04.408033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1309/2611] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:04.415913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f5e9-32d3-4c7f-8353-33b03e3284c0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:04.416191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f5e9-32d3-4c7f-8353-33b03e3284c0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:04.416460 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f5e9-32d3-4c7f-8353-33b03e3284c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:04.435947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e641f5e9-32d3-4c7f-8353-33b03e3284c0 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:04.437148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e641f5e9-32d3-4c7f-8353-33b03e3284c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:04.445562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1303/2612] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:04.449758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423e9d9-f7a3-458c-a19b-ce9bf8f93439 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:04.459901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423e9d9-f7a3-458c-a19b-ce9bf8f93439 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:04.465129 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1427b-19a6-4227-86a9-bac88aa07d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:04.465757 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1427b-19a6-4227-86a9-bac88aa07d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:04.466094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1427b-19a6-4227-86a9-bac88aa07d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:04.466372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1427b-19a6-4227-86a9-bac88aa07d41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:04.467076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1427b-19a6-4227-86a9-bac88aa07d41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:37:04.467390 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1427b-19a6-4227-86a9-bac88aa07d41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:04.469007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1427b-19a6-4227-86a9-bac88aa07d41 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:04.480950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1310/2613] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:37:04.485184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641f5e9-32d3-4c7f-8353-33b03e3284c0 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:04.494512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641f5e9-32d3-4c7f-8353-33b03e3284c0 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:04.499301 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14d2b5-723d-4aac-ac4a-d017da63d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:04.499898 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14d2b5-723d-4aac-ac4a-d017da63d344 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:04.500179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14d2b5-723d-4aac-ac4a-d017da63d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:04.500453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14d2b5-723d-4aac-ac4a-d017da63d344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:04.501074 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14d2b5-723d-4aac-ac4a-d017da63d344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:04.503039 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14d2b5-723d-4aac-ac4a-d017da63d344 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:04.673334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1304/2614] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:04 2022] POST /identity/v3/projects => generated 403 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:37:04.677467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1427b-19a6-4227-86a9-bac88aa07d41 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:04.688550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1427b-19a6-4227-86a9-bac88aa07d41 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:04.747325 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c209d5-37cb-4641-b313-0a1b20c6f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:04.747924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c209d5-37cb-4641-b313-0a1b20c6f33a admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:04.748203 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c209d5-37cb-4641-b313-0a1b20c6f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:04.748472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c209d5-37cb-4641-b313-0a1b20c6f33a admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:04.749095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c209d5-37cb-4641-b313-0a1b20c6f33a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:04.751045 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c209d5-37cb-4641-b313-0a1b20c6f33a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:04.924978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1311/2615] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:04 2022] POST /identity/v3/users => generated 302 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:04.927386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14d2b5-723d-4aac-ac4a-d017da63d344 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:04.938448 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14d2b5-723d-4aac-ac4a-d017da63d344 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:04.943105 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e08a8-fb8d-4dc8-9516-201188ff77b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:04.943832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e08a8-fb8d-4dc8-9516-201188ff77b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:04.944036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e08a8-fb8d-4dc8-9516-201188ff77b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:04.944303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e08a8-fb8d-4dc8-9516-201188ff77b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:04.945594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e08a8-fb8d-4dc8-9516-201188ff77b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:04.947232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e08a8-fb8d-4dc8-9516-201188ff77b1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:04.957398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1305/2616] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:37:04.961981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c209d5-37cb-4641-b313-0a1b20c6f33a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:04.972286 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c209d5-37cb-4641-b313-0a1b20c6f33a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:04.977835 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b260fa-1c84-4ee2-bb41-84e539519158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:04.984562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b260fa-1c84-4ee2-bb41-84e539519158 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:04.984562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b260fa-1c84-4ee2-bb41-84e539519158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:04.984562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b260fa-1c84-4ee2-bb41-84e539519158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:04.984562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b260fa-1c84-4ee2-bb41-84e539519158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:04.984562 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b260fa-1c84-4ee2-bb41-84e539519158 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:04.993580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1312/2617] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:04.998098 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e08a8-fb8d-4dc8-9516-201188ff77b1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:05.008739 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e08a8-fb8d-4dc8-9516-201188ff77b1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:05.013689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26389df5-5a16-450e-98c0-6f7b2f878661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:05.014382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26389df5-5a16-450e-98c0-6f7b2f878661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:05.014662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26389df5-5a16-450e-98c0-6f7b2f878661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:05.014938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26389df5-5a16-450e-98c0-6f7b2f878661 admin admin] PATH_INFO: `/v3/projects/90f4e63261dd41bdbbab8dbcd52ed605/users/59a27d121f4a4c2c894ad8d8aad4dc35/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:05.017215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:05.030478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26389df5-5a16-450e-98c0-6f7b2f878661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f4e63261dd41bdbbab8dbcd52ed605, user_id=59a27d121f4a4c2c894ad8d8aad4dc35, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:05.033136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26389df5-5a16-450e-98c0-6f7b2f878661 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:05.141406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1306/2618] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:04 2022] PUT /identity/v3/projects/90f4e63261dd41bdbbab8dbcd52ed605/users/59a27d121f4a4c2c894ad8d8aad4dc35/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:37:05.146743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afadaf-6dd6-40c4-91f4-4b848ce0eef3 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:05.147071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afadaf-6dd6-40c4-91f4-4b848ce0eef3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:05.147384 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afadaf-6dd6-40c4-91f4-4b848ce0eef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:05.156130 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:05.169374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b6afadaf-6dd6-40c4-91f4-4b848ce0eef3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:05.172478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b6afadaf-6dd6-40c4-91f4-4b848ce0eef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:05.237427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1313/2619] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:05.248782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26389df5-5a16-450e-98c0-6f7b2f878661 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:05.250564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-26389df5-5a16-450e-98c0-6f7b2f878661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:05.310402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26389df5-5a16-450e-98c0-6f7b2f878661 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:05.314147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15bb71-a819-4dda-bec7-f7208bcfb2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:05.314765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15bb71-a819-4dda-bec7-f7208bcfb2eb service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:05.315056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15bb71-a819-4dda-bec7-f7208bcfb2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:05.315517 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15bb71-a819-4dda-bec7-f7208bcfb2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:05.326140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15bb71-a819-4dda-bec7-f7208bcfb2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:05.327914 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15bb71-a819-4dda-bec7-f7208bcfb2eb service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:05.338553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1307/2620] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:07.004282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5bc4a-fcd2-4e87-aa05-9ebdaf9a2c10 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:07.004843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5bc4a-fcd2-4e87-aa05-9ebdaf9a2c10 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:07.004843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5bc4a-fcd2-4e87-aa05-9ebdaf9a2c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:07.036658 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-cdf5bc4a-fcd2-4e87-aa05-9ebdaf9a2c10 None None] MFA Rules not processed for user `59a27d121f4a4c2c894ad8d8aad4dc35`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:07.042563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cdf5bc4a-fcd2-4e87-aa05-9ebdaf9a2c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:07.105996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1314/2621] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:07 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:37:07.110467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15bb71-a819-4dda-bec7-f7208bcfb2eb service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:07.121085 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15bb71-a819-4dda-bec7-f7208bcfb2eb service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:07.125613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84557e-da04-42c0-9540-7c65974b2a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:07.126235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84557e-da04-42c0-9540-7c65974b2a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:07.126518 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84557e-da04-42c0-9540-7c65974b2a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:07.126792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84557e-da04-42c0-9540-7c65974b2a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:07.127412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84557e-da04-42c0-9540-7c65974b2a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:07.129325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84557e-da04-42c0-9540-7c65974b2a62 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:07.262252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1308/2622] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:07 2022] POST /identity/v3/projects => generated 403 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:07.266501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5bc4a-fcd2-4e87-aa05-9ebdaf9a2c10 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:07.277564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5bc4a-fcd2-4e87-aa05-9ebdaf9a2c10 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:07.334941 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bb358-fd4d-48c9-bbdb-d3028bfb497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:07.338862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bb358-fd4d-48c9-bbdb-d3028bfb497f admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:07.338862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bb358-fd4d-48c9-bbdb-d3028bfb497f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:07.338862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bb358-fd4d-48c9-bbdb-d3028bfb497f admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:07.338862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bb358-fd4d-48c9-bbdb-d3028bfb497f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:07.338862 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bb358-fd4d-48c9-bbdb-d3028bfb497f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:07.463189 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1315/2623] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:07 2022] POST /identity/v3/users => generated 308 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:07.464968 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84557e-da04-42c0-9540-7c65974b2a62 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:07.475771 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84557e-da04-42c0-9540-7c65974b2a62 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:07.482369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93e11b-b6fe-4365-a716-468e4e274442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:07.482974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93e11b-b6fe-4365-a716-468e4e274442 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:07.483252 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93e11b-b6fe-4365-a716-468e4e274442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:07.483537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93e11b-b6fe-4365-a716-468e4e274442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:07.484181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93e11b-b6fe-4365-a716-468e4e274442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:07.485767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93e11b-b6fe-4365-a716-468e4e274442 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:07.501111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1309/2624] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:07.504278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bb358-fd4d-48c9-bbdb-d3028bfb497f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:07.513466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bb358-fd4d-48c9-bbdb-d3028bfb497f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:07.518498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a02813-b484-448d-b30b-ec7576cc5128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:07.519206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a02813-b484-448d-b30b-ec7576cc5128 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:07.519491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a02813-b484-448d-b30b-ec7576cc5128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:07.519774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a02813-b484-448d-b30b-ec7576cc5128 admin admin] PATH_INFO: `/v3/projects/8d27f6cd097c46fab1c3c8cd0c3e7aac/users/ccfe6fe1d50242fbb8fda8e21844ed4f/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:07.536794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a02813-b484-448d-b30b-ec7576cc5128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d27f6cd097c46fab1c3c8cd0c3e7aac, user_id=ccfe6fe1d50242fbb8fda8e21844ed4f, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:07.539665 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a02813-b484-448d-b30b-ec7576cc5128 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:07.666851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:07.672852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1316/2625] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:07 2022] PUT /identity/v3/projects/8d27f6cd097c46fab1c3c8cd0c3e7aac/users/ccfe6fe1d50242fbb8fda8e21844ed4f/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:07.676837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93e11b-b6fe-4365-a716-468e4e274442 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:07.686893 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93e11b-b6fe-4365-a716-468e4e274442 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:07.742232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc98d0-6213-4cf7-9649-a50dbd8321d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:07.742838 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc98d0-6213-4cf7-9649-a50dbd8321d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:07.743118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc98d0-6213-4cf7-9649-a50dbd8321d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:07.743451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc98d0-6213-4cf7-9649-a50dbd8321d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:07.744104 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc98d0-6213-4cf7-9649-a50dbd8321d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:07.745700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc98d0-6213-4cf7-9649-a50dbd8321d0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:07.758633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1310/2626] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:37:07.762206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a02813-b484-448d-b30b-ec7576cc5128 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:07.771086 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a02813-b484-448d-b30b-ec7576cc5128 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:07.776230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c799cec-2f5c-41d1-ab3a-09e9cf88249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:07.776884 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c799cec-2f5c-41d1-ab3a-09e9cf88249e admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:07.777161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c799cec-2f5c-41d1-ab3a-09e9cf88249e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:07.777436 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c799cec-2f5c-41d1-ab3a-09e9cf88249e admin admin] PATH_INFO: `/v3/projects/8d27f6cd097c46fab1c3c8cd0c3e7aac/users/ccfe6fe1d50242fbb8fda8e21844ed4f/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:07.779413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c799cec-2f5c-41d1-ab3a-09e9cf88249e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d27f6cd097c46fab1c3c8cd0c3e7aac, user_id=ccfe6fe1d50242fbb8fda8e21844ed4f, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:07.782122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c799cec-2f5c-41d1-ab3a-09e9cf88249e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:07.794918 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1317/2627] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:07 2022] PUT /identity/v3/projects/8d27f6cd097c46fab1c3c8cd0c3e7aac/users/ccfe6fe1d50242fbb8fda8e21844ed4f/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:09.271790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc0b73-213b-4f38-82bc-d1153def1e4b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:09.272265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc0b73-213b-4f38-82bc-d1153def1e4b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:09.272379 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc0b73-213b-4f38-82bc-d1153def1e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:09.310153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cabc0b73-213b-4f38-82bc-d1153def1e4b None None] MFA Rules not processed for user `ccfe6fe1d50242fbb8fda8e21844ed4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:09.311447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cabc0b73-213b-4f38-82bc-d1153def1e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:09.362237 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:09.394862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1311/2628] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:09 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:11.195389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c799cec-2f5c-41d1-ab3a-09e9cf88249e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:11.263702 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c799cec-2f5c-41d1-ab3a-09e9cf88249e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:11.268354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:11.268986 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:11.273079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:11.273079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] PATH_INFO: `/v3/users/59a27d121f4a4c2c894ad8d8aad4dc35` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:11.273079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a27d121f4a4c2c894ad8d8aad4dc35)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:11.273079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:11.476033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a27d121f4a4c2c894ad8d8aad4dc35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:11.476718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a27d121f4a4c2c894ad8d8aad4dc35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:11.477331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a27d121f4a4c2c894ad8d8aad4dc35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:11.485416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a27d121f4a4c2c894ad8d8aad4dc35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:11.497737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a27d121f4a4c2c894ad8d8aad4dc35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:11.617107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1318/2629] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:11 2022] DELETE /identity/v3/users/59a27d121f4a4c2c894ad8d8aad4dc35 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:11.935940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc0b73-213b-4f38-82bc-d1153def1e4b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:11.952364 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc0b73-213b-4f38-82bc-d1153def1e4b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:12.012927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:12.013561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:12.013843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:12.014144 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] PATH_INFO: `/v3/projects/90f4e63261dd41bdbbab8dbcd52ed605` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:12.015274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f4e63261dd41bdbbab8dbcd52ed605)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:12.017232 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:12.135110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f4e63261dd41bdbbab8dbcd52ed605', 'request_id': 'req-8e3225fd-d628-4f11-87ff-662eb5516e38'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.140889 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f4e63261dd41bdbbab8dbcd52ed605', 'request_id': 'req-8e3225fd-d628-4f11-87ff-662eb5516e38'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.147222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f4e63261dd41bdbbab8dbcd52ed605', 'request_id': 'req-8e3225fd-d628-4f11-87ff-662eb5516e38'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.262925 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] The token cache is being invalidate because project 90f4e63261dd41bdbbab8dbcd52ed605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:12.263215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.263837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.265495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1312/2630] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:11 2022] DELETE /identity/v3/projects/90f4e63261dd41bdbbab8dbcd52ed605 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:12.269755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:12.271241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:12.285059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf826359-4126-4e59-bb5a-e8f28a228602 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:12.341848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:12.342519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:12.342800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:12.343072 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] PATH_INFO: `/v3/users/ccfe6fe1d50242fbb8fda8e21844ed4f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:12.344209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccfe6fe1d50242fbb8fda8e21844ed4f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:12.346177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:12.350644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:12.610951 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccfe6fe1d50242fbb8fda8e21844ed4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.611621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccfe6fe1d50242fbb8fda8e21844ed4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.612248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccfe6fe1d50242fbb8fda8e21844ed4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.620404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccfe6fe1d50242fbb8fda8e21844ed4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.636336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-eedb3327-143a-4c3c-9689-0aad4f76045e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccfe6fe1d50242fbb8fda8e21844ed4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:12.737621 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1319/2631] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:12 2022] DELETE /identity/v3/users/ccfe6fe1d50242fbb8fda8e21844ed4f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:13.041980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:13.043681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:13.059431 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3225fd-d628-4f11-87ff-662eb5516e38 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:13.127286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:13.127921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:13.128197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:13.128466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] PATH_INFO: `/v3/projects/8d27f6cd097c46fab1c3c8cd0c3e7aac` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:13.129599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d27f6cd097c46fab1c3c8cd0c3e7aac)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:13.131549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:13.138983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:13.274607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d27f6cd097c46fab1c3c8cd0c3e7aac', 'request_id': 'req-ada91755-7092-499e-8c52-517f322bc798'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:13.280120 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d27f6cd097c46fab1c3c8cd0c3e7aac', 'request_id': 'req-ada91755-7092-499e-8c52-517f322bc798'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:13.286508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d27f6cd097c46fab1c3c8cd0c3e7aac', 'request_id': 'req-ada91755-7092-499e-8c52-517f322bc798'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:13.411303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] The token cache is being invalidate because project 8d27f6cd097c46fab1c3c8cd0c3e7aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:13.411595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:13.412226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ada91755-7092-499e-8c52-517f322bc798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:13.413736 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1313/2632] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:13 2022] DELETE /identity/v3/projects/8d27f6cd097c46fab1c3c8cd0c3e7aac => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:13.419232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0ae6e-32b0-4474-953c-0aa5d60650c2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:13.420116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0ae6e-32b0-4474-953c-0aa5d60650c2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:13.420550 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0ae6e-32b0-4474-953c-0aa5d60650c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:13.442707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-dcd0ae6e-32b0-4474-953c-0aa5d60650c2 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:13.443949 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dcd0ae6e-32b0-4474-953c-0aa5d60650c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:13.504223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1320/2633] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:13.512876 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30200deb-a68d-454c-8f61-05eb7e2e8bcd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:13.513286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30200deb-a68d-454c-8f61-05eb7e2e8bcd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:13.513561 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30200deb-a68d-454c-8f61-05eb7e2e8bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:13.533681 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-30200deb-a68d-454c-8f61-05eb7e2e8bcd None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:13.534910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-30200deb-a68d-454c-8f61-05eb7e2e8bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:13.543350 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1314/2634] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:13.547571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0ae6e-32b0-4474-953c-0aa5d60650c2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:13.557427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0ae6e-32b0-4474-953c-0aa5d60650c2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:13.563042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de0c3d-9cd6-40d0-b992-2724d01c11d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:13.563679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de0c3d-9cd6-40d0-b992-2724d01c11d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:13.563964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de0c3d-9cd6-40d0-b992-2724d01c11d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:13.564242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de0c3d-9cd6-40d0-b992-2724d01c11d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:13.564925 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de0c3d-9cd6-40d0-b992-2724d01c11d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:37:13.565230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de0c3d-9cd6-40d0-b992-2724d01c11d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:13.567013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de0c3d-9cd6-40d0-b992-2724d01c11d7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:13.579738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1321/2635] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:37:13.584183 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30200deb-a68d-454c-8f61-05eb7e2e8bcd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:13.593644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30200deb-a68d-454c-8f61-05eb7e2e8bcd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:13.598413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c22ad1-67cf-4f22-82aa-5ec7b84f15f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:13.599013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c22ad1-67cf-4f22-82aa-5ec7b84f15f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:13.599295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c22ad1-67cf-4f22-82aa-5ec7b84f15f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:13.599571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c22ad1-67cf-4f22-82aa-5ec7b84f15f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:13.600197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c22ad1-67cf-4f22-82aa-5ec7b84f15f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:13.602278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c22ad1-67cf-4f22-82aa-5ec7b84f15f1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:13.747618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1315/2636] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:13 2022] POST /identity/v3/projects => generated 357 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:13.752165 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de0c3d-9cd6-40d0-b992-2724d01c11d7 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:13.763271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de0c3d-9cd6-40d0-b992-2724d01c11d7 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:13.820654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce15d-4c33-42fd-a69b-7b6ac78581e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:13.821297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce15d-4c33-42fd-a69b-7b6ac78581e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:13.821578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce15d-4c33-42fd-a69b-7b6ac78581e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:13.821851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce15d-4c33-42fd-a69b-7b6ac78581e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:13.822580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ce15d-4c33-42fd-a69b-7b6ac78581e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:13.825327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ce15d-4c33-42fd-a69b-7b6ac78581e8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:13.952341 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:13.957712 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1322/2637] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:13 2022] POST /identity/v3/users => generated 279 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:13.962081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c22ad1-67cf-4f22-82aa-5ec7b84f15f1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:13.972368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c22ad1-67cf-4f22-82aa-5ec7b84f15f1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:13.977050 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511088b1-5574-4c59-aad7-3757be3047c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:13.977641 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511088b1-5574-4c59-aad7-3757be3047c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:13.978011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511088b1-5574-4c59-aad7-3757be3047c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:13.978225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511088b1-5574-4c59-aad7-3757be3047c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:13.978899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511088b1-5574-4c59-aad7-3757be3047c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:13.980484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511088b1-5574-4c59-aad7-3757be3047c9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:13.990910 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1316/2638] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:13.994945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce15d-4c33-42fd-a69b-7b6ac78581e8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:14.005419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce15d-4c33-42fd-a69b-7b6ac78581e8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:14.010346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ed2bb-b326-42e9-ae38-3743a92b223b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:14.010932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ed2bb-b326-42e9-ae38-3743a92b223b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:14.011209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ed2bb-b326-42e9-ae38-3743a92b223b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:14.011478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ed2bb-b326-42e9-ae38-3743a92b223b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:14.012541 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ed2bb-b326-42e9-ae38-3743a92b223b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:14.014181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ed2bb-b326-42e9-ae38-3743a92b223b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:14.026871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1323/2639] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:37:14.029341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511088b1-5574-4c59-aad7-3757be3047c9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:14.039334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511088b1-5574-4c59-aad7-3757be3047c9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:14.044446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a8f58-aa80-4fca-9398-acd508069cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:14.045092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a8f58-aa80-4fca-9398-acd508069cff admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:14.045369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a8f58-aa80-4fca-9398-acd508069cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:14.045646 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a8f58-aa80-4fca-9398-acd508069cff admin admin] PATH_INFO: `/v3/projects/c55e2d4d5d5e447ebef58da23afa6068/users/bb69279909cd4ff5bf1a47b39284b942/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:14.060920 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a8f58-aa80-4fca-9398-acd508069cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55e2d4d5d5e447ebef58da23afa6068, user_id=bb69279909cd4ff5bf1a47b39284b942, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:14.066515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a8f58-aa80-4fca-9398-acd508069cff admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:14.175330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1317/2640] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:14 2022] PUT /identity/v3/projects/c55e2d4d5d5e447ebef58da23afa6068/users/bb69279909cd4ff5bf1a47b39284b942/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:14.186438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4763415-362b-453f-af1a-66a65bab3927 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:14.186724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4763415-362b-453f-af1a-66a65bab3927 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:14.186992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4763415-362b-453f-af1a-66a65bab3927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:14.214215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f4763415-362b-453f-af1a-66a65bab3927 None None] MFA Rules not processed for user `bb69279909cd4ff5bf1a47b39284b942`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:14.215993 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f4763415-362b-453f-af1a-66a65bab3927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:14.283680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1324/2641] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:14.288217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a8f58-aa80-4fca-9398-acd508069cff admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:14.354297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a8f58-aa80-4fca-9398-acd508069cff admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:14.359676 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c2e23-b002-4e2f-be00-c587c807b411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:14.360328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c2e23-b002-4e2f-be00-c587c807b411 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:14.360608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c2e23-b002-4e2f-be00-c587c807b411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:14.360889 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c2e23-b002-4e2f-be00-c587c807b411 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:14.361495 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c2e23-b002-4e2f-be00-c587c807b411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:14.363445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c2e23-b002-4e2f-be00-c587c807b411 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:14.467538 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1318/2642] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:14 2022] POST /identity/v3/projects => generated 357 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:14.471845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4763415-362b-453f-af1a-66a65bab3927 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:14.483691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4763415-362b-453f-af1a-66a65bab3927 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:14.543135 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2767a-aac2-440a-a712-5ff2458fe025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:14.544326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2767a-aac2-440a-a712-5ff2458fe025 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:14.544603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2767a-aac2-440a-a712-5ff2458fe025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:14.544871 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2767a-aac2-440a-a712-5ff2458fe025 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:14.545497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2767a-aac2-440a-a712-5ff2458fe025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:14.547456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2767a-aac2-440a-a712-5ff2458fe025 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:14.677105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1325/2643] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:14 2022] POST /identity/v3/users => generated 285 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:14.681282 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c2e23-b002-4e2f-be00-c587c807b411 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:14.691819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c2e23-b002-4e2f-be00-c587c807b411 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:14.696459 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbc736-c078-4e29-b50b-438abffe463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:14.697076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbc736-c078-4e29-b50b-438abffe463c admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:14.697344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbc736-c078-4e29-b50b-438abffe463c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:14.697619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbc736-c078-4e29-b50b-438abffe463c admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:14.698382 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fbc736-c078-4e29-b50b-438abffe463c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:14.700638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fbc736-c078-4e29-b50b-438abffe463c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:14.711429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1319/2644] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:14.716337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2767a-aac2-440a-a712-5ff2458fe025 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:14.726844 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2767a-aac2-440a-a712-5ff2458fe025 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:14.731736 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77aa86-ae94-4da4-9921-c41e386cd1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:14.732450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77aa86-ae94-4da4-9921-c41e386cd1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:14.732733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77aa86-ae94-4da4-9921-c41e386cd1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:14.733010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77aa86-ae94-4da4-9921-c41e386cd1b1 admin admin] PATH_INFO: `/v3/projects/f90bc4211e94430a90c36273dbdc6dc3/users/209e903b9b484224809c43e2e28e36ed/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:14.748481 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77aa86-ae94-4da4-9921-c41e386cd1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90bc4211e94430a90c36273dbdc6dc3, user_id=209e903b9b484224809c43e2e28e36ed, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:14.751365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77aa86-ae94-4da4-9921-c41e386cd1b1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:14.863461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1326/2645] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:14 2022] PUT /identity/v3/projects/f90bc4211e94430a90c36273dbdc6dc3/users/209e903b9b484224809c43e2e28e36ed/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:14.867720 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbc736-c078-4e29-b50b-438abffe463c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:14.874564 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:14.878643 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbc736-c078-4e29-b50b-438abffe463c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:14.935878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b4d0e-d922-4ca9-af97-2adda73f4edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:14.936490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b4d0e-d922-4ca9-af97-2adda73f4edd admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:14.936765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b4d0e-d922-4ca9-af97-2adda73f4edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:14.937036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b4d0e-d922-4ca9-af97-2adda73f4edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:14.937688 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b4d0e-d922-4ca9-af97-2adda73f4edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:14.939444 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b4d0e-d922-4ca9-af97-2adda73f4edd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:14.954911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1320/2646] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:14.956121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77aa86-ae94-4da4-9921-c41e386cd1b1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:14.967346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77aa86-ae94-4da4-9921-c41e386cd1b1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:14.972330 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac4897-ff94-4f2b-95da-1fc2b2774f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:14.973211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac4897-ff94-4f2b-95da-1fc2b2774f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:14.973494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac4897-ff94-4f2b-95da-1fc2b2774f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:14.973804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac4897-ff94-4f2b-95da-1fc2b2774f04 admin admin] PATH_INFO: `/v3/projects/f90bc4211e94430a90c36273dbdc6dc3/users/209e903b9b484224809c43e2e28e36ed/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:14.975797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac4897-ff94-4f2b-95da-1fc2b2774f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90bc4211e94430a90c36273dbdc6dc3, user_id=209e903b9b484224809c43e2e28e36ed, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:14.978615 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac4897-ff94-4f2b-95da-1fc2b2774f04 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:14.991791 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1327/2647] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:14 2022] PUT /identity/v3/projects/f90bc4211e94430a90c36273dbdc6dc3/users/209e903b9b484224809c43e2e28e36ed/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:15.003900 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982abfa5-8871-4f75-a102-247d838275ea None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:15.004186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982abfa5-8871-4f75-a102-247d838275ea None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:15.004450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982abfa5-8871-4f75-a102-247d838275ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:15.033055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-982abfa5-8871-4f75-a102-247d838275ea None None] MFA Rules not processed for user `209e903b9b484224809c43e2e28e36ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:15.034321 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-982abfa5-8871-4f75-a102-247d838275ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:15.101693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1321/2648] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:15 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:15.115821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac4897-ff94-4f2b-95da-1fc2b2774f04 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:15.117630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e2ac4897-ff94-4f2b-95da-1fc2b2774f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:15.155385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:15.179842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac4897-ff94-4f2b-95da-1fc2b2774f04 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:15.184824 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75262315-ef49-47dc-8ad8-7aca5fd1d06f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:15.185457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75262315-ef49-47dc-8ad8-7aca5fd1d06f service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:15.185745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75262315-ef49-47dc-8ad8-7aca5fd1d06f service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:15.186061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75262315-ef49-47dc-8ad8-7aca5fd1d06f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:15.195983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75262315-ef49-47dc-8ad8-7aca5fd1d06f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:15.197434 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75262315-ef49-47dc-8ad8-7aca5fd1d06f service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:15.207095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1328/2649] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:15.470948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982abfa5-8871-4f75-a102-247d838275ea None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:15.481060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982abfa5-8871-4f75-a102-247d838275ea None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:15.484711 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041531c-7f97-4e11-86c0-0d4015da41df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:15.485287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041531c-7f97-4e11-86c0-0d4015da41df service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:15.485563 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041531c-7f97-4e11-86c0-0d4015da41df service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:15.485834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041531c-7f97-4e11-86c0-0d4015da41df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:15.543767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041531c-7f97-4e11-86c0-0d4015da41df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:15.545212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041531c-7f97-4e11-86c0-0d4015da41df service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:15.555195 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1322/2650] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:25.327545 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=12 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:25.550340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:26.435754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018d672-4f61-442f-a3bf-ba6aecaa9651 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:26.436211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018d672-4f61-442f-a3bf-ba6aecaa9651 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:26.436535 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018d672-4f61-442f-a3bf-ba6aecaa9651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:26.456588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f018d672-4f61-442f-a3bf-ba6aecaa9651 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:26.457962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f018d672-4f61-442f-a3bf-ba6aecaa9651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:26.522997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1329/2651] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:37:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:26.546341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041531c-7f97-4e11-86c0-0d4015da41df service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:26.547806 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5041531c-7f97-4e11-86c0-0d4015da41df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:26.618713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041531c-7f97-4e11-86c0-0d4015da41df service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:26.623693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c692d-cfaf-4427-8281-bc51e92409d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:26.624319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c692d-cfaf-4427-8281-bc51e92409d5 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:26.624591 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c692d-cfaf-4427-8281-bc51e92409d5 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:26.624859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c692d-cfaf-4427-8281-bc51e92409d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:26.633491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c692d-cfaf-4427-8281-bc51e92409d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:26.635372 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c692d-cfaf-4427-8281-bc51e92409d5 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:26.643915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1323/2652] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:37:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:37:27.577684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627eb421-cca8-44d9-b2bd-327feffb5af6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:27.578159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627eb421-cca8-44d9-b2bd-327feffb5af6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:27.578250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627eb421-cca8-44d9-b2bd-327feffb5af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:27.598528 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-627eb421-cca8-44d9-b2bd-327feffb5af6 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:27.599778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-627eb421-cca8-44d9-b2bd-327feffb5af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:27.607080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1330/2653] 10.222.0.83 () {58 vars in 967 bytes} [Wed Aug 24 11:37:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:27.619636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c692d-cfaf-4427-8281-bc51e92409d5 service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:27.630726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c692d-cfaf-4427-8281-bc51e92409d5 service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:27.635844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187aa6c-0842-4dd2-a857-cfee9afb300d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:27.636387 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187aa6c-0842-4dd2-a857-cfee9afb300d service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:27.636660 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187aa6c-0842-4dd2-a857-cfee9afb300d service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:27.636928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187aa6c-0842-4dd2-a857-cfee9afb300d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:27.645713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187aa6c-0842-4dd2-a857-cfee9afb300d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:27.647365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187aa6c-0842-4dd2-a857-cfee9afb300d service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:27.655354 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1324/2654] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:37:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:37:28.387632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02410013-3d3d-4ae9-a32e-cafb01a6cbdf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:28.387918 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02410013-3d3d-4ae9-a32e-cafb01a6cbdf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:28.388198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02410013-3d3d-4ae9-a32e-cafb01a6cbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:28.408372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-02410013-3d3d-4ae9-a32e-cafb01a6cbdf None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:28.410167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-02410013-3d3d-4ae9-a32e-cafb01a6cbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:28.417385 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1331/2655] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:37:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:28.435072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187aa6c-0842-4dd2-a857-cfee9afb300d service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:28.445763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187aa6c-0842-4dd2-a857-cfee9afb300d service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:28.450836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e4c48-97e9-4162-822e-143a4b4f2ca1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:28.451468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e4c48-97e9-4162-822e-143a4b4f2ca1 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:28.451744 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e4c48-97e9-4162-822e-143a4b4f2ca1 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:28.452016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e4c48-97e9-4162-822e-143a4b4f2ca1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:28.460878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e4c48-97e9-4162-822e-143a4b4f2ca1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:28.462423 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e4c48-97e9-4162-822e-143a4b4f2ca1 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:28.470466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1325/2656] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:37:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 11:37:29.108577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9160b-a83c-49af-a401-6dabdce3ab90 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:29.109067 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9160b-a83c-49af-a401-6dabdce3ab90 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:29.109197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9160b-a83c-49af-a401-6dabdce3ab90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:29.129089 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4bf9160b-a83c-49af-a401-6dabdce3ab90 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:29.130339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4bf9160b-a83c-49af-a401-6dabdce3ab90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:29.137832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1332/2657] 10.222.0.83 () {58 vars in 967 bytes} [Wed Aug 24 11:37:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:29.149961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e4c48-97e9-4162-822e-143a4b4f2ca1 service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:29.160734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e4c48-97e9-4162-822e-143a4b4f2ca1 service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:29.165732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e26661-4489-4cf1-a1e8-61e7d05ac189 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:29.166365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e26661-4489-4cf1-a1e8-61e7d05ac189 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:29.166647 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e26661-4489-4cf1-a1e8-61e7d05ac189 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:29.166948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e26661-4489-4cf1-a1e8-61e7d05ac189 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:29.175685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e26661-4489-4cf1-a1e8-61e7d05ac189 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:29.177269 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e26661-4489-4cf1-a1e8-61e7d05ac189 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:29.185539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1326/2658] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:37:32.316206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9160b-a83c-49af-a401-6dabdce3ab90 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:32.317656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4bf9160b-a83c-49af-a401-6dabdce3ab90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:32.389493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9160b-a83c-49af-a401-6dabdce3ab90 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:32.394741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:32.395376 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:32.395663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:32.395931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] PATH_INFO: `/v3/users/effc45029842477e92552609037f2e35` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:32.397083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=effc45029842477e92552609037f2e35)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:32.399145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:32.677398 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'effc45029842477e92552609037f2e35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:32.677995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'effc45029842477e92552609037f2e35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:32.678795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'effc45029842477e92552609037f2e35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:32.685747 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'effc45029842477e92552609037f2e35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:32.697566 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'effc45029842477e92552609037f2e35'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:32.772651 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1333/2659] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:32 2022] DELETE /identity/v3/users/effc45029842477e92552609037f2e35 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:32.777132 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6707f-da0e-4176-b6de-278b4760c0a3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:32.777613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6707f-da0e-4176-b6de-278b4760c0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:32.777728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6707f-da0e-4176-b6de-278b4760c0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:32.796915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-dab6707f-da0e-4176-b6de-278b4760c0a3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:32.798197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dab6707f-da0e-4176-b6de-278b4760c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:32.856800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1327/2660] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:32.867167 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:32.868584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:32.875254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:32.927648 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ec1a4-6a4d-486c-bfea-942993ee46bc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:32.931370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6e7f-4a16-4c86-a347-e133e9c4dca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:32.931982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6e7f-4a16-4c86-a347-e133e9c4dca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:32.932264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6e7f-4a16-4c86-a347-e133e9c4dca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:32.932535 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6e7f-4a16-4c86-a347-e133e9c4dca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:32.941917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac6e7f-4a16-4c86-a347-e133e9c4dca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:32.943371 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac6e7f-4a16-4c86-a347-e133e9c4dca9 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:32.953469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1334/2661] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:33.191491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6707f-da0e-4176-b6de-278b4760c0a3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:33.192913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dab6707f-da0e-4176-b6de-278b4760c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:33.208609 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6707f-da0e-4176-b6de-278b4760c0a3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:33.213586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:33.214275 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.214551 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.214826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] PATH_INFO: `/v3/projects/b4d27b24d8e5430c93890088fb6386db` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.216111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4d27b24d8e5430c93890088fb6386db)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:33.218167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:33.332182 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:33.335235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4d27b24d8e5430c93890088fb6386db', 'request_id': 'req-3618b821-4230-4c85-b58b-1e144588d8a4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:33.340730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4d27b24d8e5430c93890088fb6386db', 'request_id': 'req-3618b821-4230-4c85-b58b-1e144588d8a4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:33.348853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4d27b24d8e5430c93890088fb6386db', 'request_id': 'req-3618b821-4230-4c85-b58b-1e144588d8a4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:33.443030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] The token cache is being invalidate because project b4d27b24d8e5430c93890088fb6386db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:33.444818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:33.444818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3618b821-4230-4c85-b58b-1e144588d8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:33.445949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1328/2662] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:33 2022] DELETE /identity/v3/projects/b4d27b24d8e5430c93890088fb6386db => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:37:33.451144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b954d-472c-49db-9e17-50267f8244be None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.451144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b954d-472c-49db-9e17-50267f8244be None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.451760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b954d-472c-49db-9e17-50267f8244be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.472245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2b6b954d-472c-49db-9e17-50267f8244be None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:33.473629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2b6b954d-472c-49db-9e17-50267f8244be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:33.538023 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1335/2663] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:33.545568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a511a7-3a9e-467d-b51e-37099a0ccfda None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.545975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a511a7-3a9e-467d-b51e-37099a0ccfda None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.546157 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a511a7-3a9e-467d-b51e-37099a0ccfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.566834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-23a511a7-3a9e-467d-b51e-37099a0ccfda None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:33.568135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-23a511a7-3a9e-467d-b51e-37099a0ccfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:33.576536 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1329/2664] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:33.582582 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b954d-472c-49db-9e17-50267f8244be None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:33.592706 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b954d-472c-49db-9e17-50267f8244be None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:33.598173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19eee8-4be4-4f8c-8f4f-71c3e879b659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:33.598785 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19eee8-4be4-4f8c-8f4f-71c3e879b659 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.599069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19eee8-4be4-4f8c-8f4f-71c3e879b659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.599401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19eee8-4be4-4f8c-8f4f-71c3e879b659 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.600056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19eee8-4be4-4f8c-8f4f-71c3e879b659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:37:33.600365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19eee8-4be4-4f8c-8f4f-71c3e879b659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:33.602013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19eee8-4be4-4f8c-8f4f-71c3e879b659 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:33.614488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1336/2665] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:37:33.619719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a511a7-3a9e-467d-b51e-37099a0ccfda None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:33.629262 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a511a7-3a9e-467d-b51e-37099a0ccfda None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:33.634289 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4cf4e-9f69-40e7-8f77-8be67105cb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:33.634884 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4cf4e-9f69-40e7-8f77-8be67105cb66 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.635167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4cf4e-9f69-40e7-8f77-8be67105cb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.635445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4cf4e-9f69-40e7-8f77-8be67105cb66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.636071 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4cf4e-9f69-40e7-8f77-8be67105cb66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:33.638012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4cf4e-9f69-40e7-8f77-8be67105cb66 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:33.741384 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1330/2666] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:33 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:33.745926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19eee8-4be4-4f8c-8f4f-71c3e879b659 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:33.757064 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19eee8-4be4-4f8c-8f4f-71c3e879b659 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:33.813802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a3384-d958-4ca1-b315-2675ab06af5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:33.814524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3384-d958-4ca1-b315-2675ab06af5e admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.814802 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3384-d958-4ca1-b315-2675ab06af5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.815069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3384-d958-4ca1-b315-2675ab06af5e admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.815715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a3384-d958-4ca1-b315-2675ab06af5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:33.817899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a3384-d958-4ca1-b315-2675ab06af5e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:33.939201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1337/2667] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:33 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:37:33.943939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4cf4e-9f69-40e7-8f77-8be67105cb66 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:33.954516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4cf4e-9f69-40e7-8f77-8be67105cb66 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:33.959336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418fcc1-0f91-4e1a-95ae-77ca8262ca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:33.959931 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418fcc1-0f91-4e1a-95ae-77ca8262ca8b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.960214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418fcc1-0f91-4e1a-95ae-77ca8262ca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.960491 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418fcc1-0f91-4e1a-95ae-77ca8262ca8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.961140 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418fcc1-0f91-4e1a-95ae-77ca8262ca8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:33.962791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418fcc1-0f91-4e1a-95ae-77ca8262ca8b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:33.973318 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1331/2668] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:33.977411 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a3384-d958-4ca1-b315-2675ab06af5e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:33.988397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a3384-d958-4ca1-b315-2675ab06af5e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:33.993041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dffd2f-065b-4dce-8a00-cf932eee75aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:33.993633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dffd2f-065b-4dce-8a00-cf932eee75aa admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.993943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dffd2f-065b-4dce-8a00-cf932eee75aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.994218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dffd2f-065b-4dce-8a00-cf932eee75aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.994936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dffd2f-065b-4dce-8a00-cf932eee75aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:33.996548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dffd2f-065b-4dce-8a00-cf932eee75aa admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:34.007249 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1338/2669] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:34.011590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418fcc1-0f91-4e1a-95ae-77ca8262ca8b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:34.021809 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418fcc1-0f91-4e1a-95ae-77ca8262ca8b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:34.026703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a3c6-1b49-4aca-b1b5-2f3c922b1d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:34.027369 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a3c6-1b49-4aca-b1b5-2f3c922b1d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:34.027651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a3c6-1b49-4aca-b1b5-2f3c922b1d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:34.027929 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a3c6-1b49-4aca-b1b5-2f3c922b1d03 admin admin] PATH_INFO: `/v3/projects/3583d4d34a634ea9a30c1f96ba70a664/users/2e40e7790fe94c388cb206aab7c94734/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:34.048395 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414a3c6-1b49-4aca-b1b5-2f3c922b1d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3583d4d34a634ea9a30c1f96ba70a664, user_id=2e40e7790fe94c388cb206aab7c94734, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:34.051273 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414a3c6-1b49-4aca-b1b5-2f3c922b1d03 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:34.125754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1332/2670] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:34 2022] PUT /identity/v3/projects/3583d4d34a634ea9a30c1f96ba70a664/users/2e40e7790fe94c388cb206aab7c94734/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:37:34.131392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d3b72-eca1-4d4b-b3a0-455d356ebf47 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:34.131707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d3b72-eca1-4d4b-b3a0-455d356ebf47 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:34.131981 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d3b72-eca1-4d4b-b3a0-455d356ebf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:34.151714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ec5d3b72-eca1-4d4b-b3a0-455d356ebf47 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:34.153015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ec5d3b72-eca1-4d4b-b3a0-455d356ebf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:34.164321 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:34.221962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1339/2671] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:34.232014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a3c6-1b49-4aca-b1b5-2f3c922b1d03 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:34.235799 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c414a3c6-1b49-4aca-b1b5-2f3c922b1d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:34.295943 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a3c6-1b49-4aca-b1b5-2f3c922b1d03 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:34.299779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c34ab-397b-4725-80df-1307c2ecc28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:34.300415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c34ab-397b-4725-80df-1307c2ecc28e service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:34.300690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c34ab-397b-4725-80df-1307c2ecc28e service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:34.300960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c34ab-397b-4725-80df-1307c2ecc28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:34.310247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c34ab-397b-4725-80df-1307c2ecc28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:34.311670 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c34ab-397b-4725-80df-1307c2ecc28e service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:34.326537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1333/2672] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:36.154748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd511880-6ee4-4939-a24d-fe6d5fa864ed None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:36.158737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd511880-6ee4-4939-a24d-fe6d5fa864ed None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:36.158737 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd511880-6ee4-4939-a24d-fe6d5fa864ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:36.185347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bd511880-6ee4-4939-a24d-fe6d5fa864ed None None] MFA Rules not processed for user `2e40e7790fe94c388cb206aab7c94734`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:36.186784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bd511880-6ee4-4939-a24d-fe6d5fa864ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:36.260146 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1340/2673] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:36.271368 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c34ab-397b-4725-80df-1307c2ecc28e service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:36.272775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6f3c34ab-397b-4725-80df-1307c2ecc28e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:36.332671 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c34ab-397b-4725-80df-1307c2ecc28e service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:36.336449 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b78591-d117-4a57-8269-72e974a3a898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:36.337052 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b78591-d117-4a57-8269-72e974a3a898 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:36.337344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b78591-d117-4a57-8269-72e974a3a898 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:36.337619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b78591-d117-4a57-8269-72e974a3a898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:36.346552 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b78591-d117-4a57-8269-72e974a3a898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:36.348042 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b78591-d117-4a57-8269-72e974a3a898 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:36.356956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1334/2674] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:38.140395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd511880-6ee4-4939-a24d-fe6d5fa864ed None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:38.141909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bd511880-6ee4-4939-a24d-fe6d5fa864ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:38.156060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd511880-6ee4-4939-a24d-fe6d5fa864ed None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:38.161188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:38.161813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:38.162163 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:38.162441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] PATH_INFO: `/v3/users/bb69279909cd4ff5bf1a47b39284b942` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:38.163569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb69279909cd4ff5bf1a47b39284b942)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:38.165495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:38.414994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb69279909cd4ff5bf1a47b39284b942'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:38.415656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb69279909cd4ff5bf1a47b39284b942'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:38.416262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb69279909cd4ff5bf1a47b39284b942'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:38.423358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb69279909cd4ff5bf1a47b39284b942'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:38.442241 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb69279909cd4ff5bf1a47b39284b942'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:38.562039 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1341/2675] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:38 2022] DELETE /identity/v3/users/bb69279909cd4ff5bf1a47b39284b942 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:37:38.567130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348486-e3ca-4174-85f1-565114fdbbe8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:38.567584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348486-e3ca-4174-85f1-565114fdbbe8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:38.567712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348486-e3ca-4174-85f1-565114fdbbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:38.587964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-49348486-e3ca-4174-85f1-565114fdbbe8 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:38.589324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-49348486-e3ca-4174-85f1-565114fdbbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:38.598975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:38.659852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1335/2676] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:38.669584 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:38.671014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:38.692201 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:38.731372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c47290-6408-4e52-907f-d15d63b058d5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:38.734901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696d48-76fc-4199-a205-6a951a6701b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:38.735514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696d48-76fc-4199-a205-6a951a6701b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:38.735795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696d48-76fc-4199-a205-6a951a6701b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:38.736070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696d48-76fc-4199-a205-6a951a6701b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:38.745139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae696d48-76fc-4199-a205-6a951a6701b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:38.746615 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae696d48-76fc-4199-a205-6a951a6701b3 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:38.756574 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1342/2677] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:39.156908 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49348486-e3ca-4174-85f1-565114fdbbe8 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:39.159056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-49348486-e3ca-4174-85f1-565114fdbbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:39.172181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49348486-e3ca-4174-85f1-565114fdbbe8 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:39.177228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:39.177895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:39.178181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:39.178458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] PATH_INFO: `/v3/projects/c55e2d4d5d5e447ebef58da23afa6068` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:39.179605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c55e2d4d5d5e447ebef58da23afa6068)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:39.181528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:39.293826 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c55e2d4d5d5e447ebef58da23afa6068', 'request_id': 'req-d42b1871-7036-43c0-993f-2f75e66b146c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.302442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c55e2d4d5d5e447ebef58da23afa6068', 'request_id': 'req-d42b1871-7036-43c0-993f-2f75e66b146c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.309083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c55e2d4d5d5e447ebef58da23afa6068', 'request_id': 'req-d42b1871-7036-43c0-993f-2f75e66b146c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.353513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a7e03-ce68-4b10-9fa5-c6ea7425aa86 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:39.354102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a7e03-ce68-4b10-9fa5-c6ea7425aa86 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:39.354102 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a7e03-ce68-4b10-9fa5-c6ea7425aa86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:39.372966 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ea6a7e03-ce68-4b10-9fa5-c6ea7425aa86 None None] MFA Rules not processed for user `2e40e7790fe94c388cb206aab7c94734`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:39.374210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ea6a7e03-ce68-4b10-9fa5-c6ea7425aa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:39.417702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] The token cache is being invalidate because project c55e2d4d5d5e447ebef58da23afa6068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:39.418044 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.418819 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.420745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1336/2678] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:39 2022] DELETE /identity/v3/projects/c55e2d4d5d5e447ebef58da23afa6068 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:39.424628 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:39.426302 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:39.440014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1343/2679] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:37:39.493864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b1871-7036-43c0-993f-2f75e66b146c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:39.499060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:39.499702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:39.500585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:39.500585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] PATH_INFO: `/v3/users/209e903b9b484224809c43e2e28e36ed` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:39.502147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] RBAC: Authorizing `identity:delete_user(user_id=209e903b9b484224809c43e2e28e36ed)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:39.503940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:39.771172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:39.776578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '209e903b9b484224809c43e2e28e36ed'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.777243 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '209e903b9b484224809c43e2e28e36ed'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.777892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '209e903b9b484224809c43e2e28e36ed'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.875061 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '209e903b9b484224809c43e2e28e36ed'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.886037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '209e903b9b484224809c43e2e28e36ed'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:39.902568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1337/2680] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:39 2022] DELETE /identity/v3/users/209e903b9b484224809c43e2e28e36ed => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:37:40.180095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a7e03-ce68-4b10-9fa5-c6ea7425aa86 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:40.181944 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ea6a7e03-ce68-4b10-9fa5-c6ea7425aa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:40.266623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a7e03-ce68-4b10-9fa5-c6ea7425aa86 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:40.335847 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38344a03-3765-4c79-8611-befbbd4c34ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:40.336558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38344a03-3765-4c79-8611-befbbd4c34ae service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:40.336905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38344a03-3765-4c79-8611-befbbd4c34ae service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:40.337270 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38344a03-3765-4c79-8611-befbbd4c34ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:40.339169 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-38344a03-3765-4c79-8611-befbbd4c34ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:40.349629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:40.352107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38344a03-3765-4c79-8611-befbbd4c34ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:40.353744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38344a03-3765-4c79-8611-befbbd4c34ae service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:40.364679 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1344/2681] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:40.482709 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:40.484294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:40.555130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e9cbf-607f-440d-81e7-c9755b48815e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:40.559765 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:40.560386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:40.560666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:40.560939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] PATH_INFO: `/v3/projects/f90bc4211e94430a90c36273dbdc6dc3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:40.562092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90bc4211e94430a90c36273dbdc6dc3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:40.564048 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:40.675582 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90bc4211e94430a90c36273dbdc6dc3', 'request_id': 'req-df805a39-9f05-4d4c-bc77-cc3ac32adb10'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:40.681528 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90bc4211e94430a90c36273dbdc6dc3', 'request_id': 'req-df805a39-9f05-4d4c-bc77-cc3ac32adb10'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:40.687901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90bc4211e94430a90c36273dbdc6dc3', 'request_id': 'req-df805a39-9f05-4d4c-bc77-cc3ac32adb10'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:40.790236 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] The token cache is being invalidate because project f90bc4211e94430a90c36273dbdc6dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:40.791030 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:40.791310 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-df805a39-9f05-4d4c-bc77-cc3ac32adb10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:40.792834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1338/2682] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:40 2022] DELETE /identity/v3/projects/f90bc4211e94430a90c36273dbdc6dc3 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:40.798761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c89b2-39ae-40e3-b0f5-3a91a087fe36 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:40.799103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c89b2-39ae-40e3-b0f5-3a91a087fe36 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:40.799458 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c89b2-39ae-40e3-b0f5-3a91a087fe36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:40.821151 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9e4c89b2-39ae-40e3-b0f5-3a91a087fe36 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:40.822349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9e4c89b2-39ae-40e3-b0f5-3a91a087fe36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:40.883337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1345/2683] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:40.891622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c2cc7-d743-4c05-b092-be55d871a4e3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:40.891911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c2cc7-d743-4c05-b092-be55d871a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:40.892832 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c2cc7-d743-4c05-b092-be55d871a4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:40.912309 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-408c2cc7-d743-4c05-b092-be55d871a4e3 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:40.913525 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-408c2cc7-d743-4c05-b092-be55d871a4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:40.921531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1339/2684] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:40.925795 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c89b2-39ae-40e3-b0f5-3a91a087fe36 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:40.935561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c89b2-39ae-40e3-b0f5-3a91a087fe36 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:40.940669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42842835-f4d1-4193-8b5f-085e3a90befe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:40.941259 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42842835-f4d1-4193-8b5f-085e3a90befe admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:40.941534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42842835-f4d1-4193-8b5f-085e3a90befe admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:40.941803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42842835-f4d1-4193-8b5f-085e3a90befe admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:40.942561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42842835-f4d1-4193-8b5f-085e3a90befe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:37:40.942866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42842835-f4d1-4193-8b5f-085e3a90befe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:40.944469 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42842835-f4d1-4193-8b5f-085e3a90befe admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:40.957184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1346/2685] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:37:40.961415 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c2cc7-d743-4c05-b092-be55d871a4e3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:40.971870 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c2cc7-d743-4c05-b092-be55d871a4e3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:40.976818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7209312-8734-4662-98e1-74c74b394a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:40.977433 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7209312-8734-4662-98e1-74c74b394a81 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:40.977827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7209312-8734-4662-98e1-74c74b394a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:40.978151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7209312-8734-4662-98e1-74c74b394a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:40.978801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7209312-8734-4662-98e1-74c74b394a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:40.980766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7209312-8734-4662-98e1-74c74b394a81 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:41.119015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1340/2686] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:40 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:37:41.123599 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42842835-f4d1-4193-8b5f-085e3a90befe admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:41.134116 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42842835-f4d1-4193-8b5f-085e3a90befe admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:41.191098 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148792c5-279e-4b83-8686-7bfcb2ed617b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:41.191714 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148792c5-279e-4b83-8686-7bfcb2ed617b admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:41.191999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148792c5-279e-4b83-8686-7bfcb2ed617b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:41.192277 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148792c5-279e-4b83-8686-7bfcb2ed617b admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:41.192948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148792c5-279e-4b83-8686-7bfcb2ed617b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:41.194965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148792c5-279e-4b83-8686-7bfcb2ed617b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:41.395050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1347/2687] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:41 2022] POST /identity/v3/users => generated 294 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:37:41.399153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7209312-8734-4662-98e1-74c74b394a81 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:41.410295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7209312-8734-4662-98e1-74c74b394a81 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:41.414895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f67f1-a38b-4db0-87d4-daee1611a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:41.415526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f67f1-a38b-4db0-87d4-daee1611a24f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:41.415805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f67f1-a38b-4db0-87d4-daee1611a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:41.416077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f67f1-a38b-4db0-87d4-daee1611a24f admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:41.416766 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f67f1-a38b-4db0-87d4-daee1611a24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:41.418416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f67f1-a38b-4db0-87d4-daee1611a24f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:41.428824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1341/2688] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:41.432926 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148792c5-279e-4b83-8686-7bfcb2ed617b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:41.443235 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148792c5-279e-4b83-8686-7bfcb2ed617b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:41.447872 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaac949-3f78-48af-aefa-095146ca0bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:41.448464 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaac949-3f78-48af-aefa-095146ca0bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:41.449710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaac949-3f78-48af-aefa-095146ca0bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:41.450096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaac949-3f78-48af-aefa-095146ca0bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:41.450790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaac949-3f78-48af-aefa-095146ca0bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:41.452435 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaac949-3f78-48af-aefa-095146ca0bd2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:41.464421 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1348/2689] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:41.468796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f67f1-a38b-4db0-87d4-daee1611a24f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:41.479526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f67f1-a38b-4db0-87d4-daee1611a24f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:41.484149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7fba4-86aa-4e8f-b1a0-a9a7f3f4faba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:41.484790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7fba4-86aa-4e8f-b1a0-a9a7f3f4faba admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:41.485072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7fba4-86aa-4e8f-b1a0-a9a7f3f4faba admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:41.485347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7fba4-86aa-4e8f-b1a0-a9a7f3f4faba admin admin] PATH_INFO: `/v3/projects/eb9aacaa61a548eda727d96ea34f0786/users/d30a97daef914ace90075f13056dc58c/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:41.489024 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:41.502685 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7fba4-86aa-4e8f-b1a0-a9a7f3f4faba admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9aacaa61a548eda727d96ea34f0786, user_id=d30a97daef914ace90075f13056dc58c, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:41.504862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7fba4-86aa-4e8f-b1a0-a9a7f3f4faba admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:41.622728 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1342/2690] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:41 2022] PUT /identity/v3/projects/eb9aacaa61a548eda727d96ea34f0786/users/d30a97daef914ace90075f13056dc58c/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:41.635079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007347b0-be1b-4abb-a04e-bca96e6fbc6e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:41.635362 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007347b0-be1b-4abb-a04e-bca96e6fbc6e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:41.635632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007347b0-be1b-4abb-a04e-bca96e6fbc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:41.665816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-007347b0-be1b-4abb-a04e-bca96e6fbc6e None None] MFA Rules not processed for user `d30a97daef914ace90075f13056dc58c`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:41.667011 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-007347b0-be1b-4abb-a04e-bca96e6fbc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:41.715395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:41.731805 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1349/2691] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:41.736467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7fba4-86aa-4e8f-b1a0-a9a7f3f4faba admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:41.926142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7fba4-86aa-4e8f-b1a0-a9a7f3f4faba admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:41.930805 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87bbca-96aa-4563-9b91-43b0ea1beb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:41.931319 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87bbca-96aa-4563-9b91-43b0ea1beb85 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:41.931598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87bbca-96aa-4563-9b91-43b0ea1beb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:41.931862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87bbca-96aa-4563-9b91-43b0ea1beb85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:41.932549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87bbca-96aa-4563-9b91-43b0ea1beb85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:41.934485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87bbca-96aa-4563-9b91-43b0ea1beb85 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:42.054290 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1343/2692] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:41 2022] POST /identity/v3/projects => generated 385 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:42.058389 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007347b0-be1b-4abb-a04e-bca96e6fbc6e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:42.069552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007347b0-be1b-4abb-a04e-bca96e6fbc6e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:42.125677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869baf4-a732-4981-9b08-58521107f2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:42.126311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869baf4-a732-4981-9b08-58521107f2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:42.126589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869baf4-a732-4981-9b08-58521107f2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:42.126859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869baf4-a732-4981-9b08-58521107f2a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:42.127485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869baf4-a732-4981-9b08-58521107f2a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:42.129407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869baf4-a732-4981-9b08-58521107f2a6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:42.219222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1350/2693] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:42 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:42.223135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87bbca-96aa-4563-9b91-43b0ea1beb85 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:42.233755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87bbca-96aa-4563-9b91-43b0ea1beb85 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:42.239211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fde78-f304-4408-abac-517abd2700a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:42.239779 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fde78-f304-4408-abac-517abd2700a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:42.240058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fde78-f304-4408-abac-517abd2700a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:42.240328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fde78-f304-4408-abac-517abd2700a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:42.241640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fde78-f304-4408-abac-517abd2700a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:42.243462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fde78-f304-4408-abac-517abd2700a0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:42.254461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1344/2694] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:42.259495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869baf4-a732-4981-9b08-58521107f2a6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:42.269803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869baf4-a732-4981-9b08-58521107f2a6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:42.274597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59634f-7997-4fd9-8c2f-c2198a59e717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:42.275256 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59634f-7997-4fd9-8c2f-c2198a59e717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:42.275536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59634f-7997-4fd9-8c2f-c2198a59e717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:42.275823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59634f-7997-4fd9-8c2f-c2198a59e717 admin admin] PATH_INFO: `/v3/projects/27057e8771f746f0bc963a55f28120e6/users/ca11c16215c84181bdb2e85ed40a2fc9/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:42.292473 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59634f-7997-4fd9-8c2f-c2198a59e717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27057e8771f746f0bc963a55f28120e6, user_id=ca11c16215c84181bdb2e85ed40a2fc9, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:42.303142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59634f-7997-4fd9-8c2f-c2198a59e717 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:42.397278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1351/2695] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:42 2022] PUT /identity/v3/projects/27057e8771f746f0bc963a55f28120e6/users/ca11c16215c84181bdb2e85ed40a2fc9/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:42.401164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fde78-f304-4408-abac-517abd2700a0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:42.413655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fde78-f304-4408-abac-517abd2700a0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:42.478019 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c773e57-136a-4b93-bbe5-ce2e92835775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:42.478539 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c773e57-136a-4b93-bbe5-ce2e92835775 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:42.478830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c773e57-136a-4b93-bbe5-ce2e92835775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:42.479092 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c773e57-136a-4b93-bbe5-ce2e92835775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:42.479732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c773e57-136a-4b93-bbe5-ce2e92835775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:42.481320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c773e57-136a-4b93-bbe5-ce2e92835775 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:42.491879 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1345/2696] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:42.496196 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59634f-7997-4fd9-8c2f-c2198a59e717 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:42.507818 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59634f-7997-4fd9-8c2f-c2198a59e717 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:42.513070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4652ea4-0fc5-404c-8bbf-98a80a7c7806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:42.513760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4652ea4-0fc5-404c-8bbf-98a80a7c7806 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:42.514074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4652ea4-0fc5-404c-8bbf-98a80a7c7806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:42.514374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4652ea4-0fc5-404c-8bbf-98a80a7c7806 admin admin] PATH_INFO: `/v3/projects/27057e8771f746f0bc963a55f28120e6/users/ca11c16215c84181bdb2e85ed40a2fc9/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:42.516538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4652ea4-0fc5-404c-8bbf-98a80a7c7806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27057e8771f746f0bc963a55f28120e6, user_id=ca11c16215c84181bdb2e85ed40a2fc9, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:42.519717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4652ea4-0fc5-404c-8bbf-98a80a7c7806 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:42.532485 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1352/2697] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:42 2022] PUT /identity/v3/projects/27057e8771f746f0bc963a55f28120e6/users/ca11c16215c84181bdb2e85ed40a2fc9/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:37:42.541462 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88293b37-319d-46b1-9e0b-2256a214b980 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:42.541756 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88293b37-319d-46b1-9e0b-2256a214b980 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:42.542145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88293b37-319d-46b1-9e0b-2256a214b980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:42.570682 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-88293b37-319d-46b1-9e0b-2256a214b980 None None] MFA Rules not processed for user `ca11c16215c84181bdb2e85ed40a2fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:42.578519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-88293b37-319d-46b1-9e0b-2256a214b980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:42.615474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:42.639843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1346/2698] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:42.650500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4652ea4-0fc5-404c-8bbf-98a80a7c7806 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:42.652225 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f4652ea4-0fc5-404c-8bbf-98a80a7c7806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:42.720690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4652ea4-0fc5-404c-8bbf-98a80a7c7806 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:42.724526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fd11d-905f-4249-8912-3090d96213d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:42.725171 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fd11d-905f-4249-8912-3090d96213d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:42.725453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fd11d-905f-4249-8912-3090d96213d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:42.725729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fd11d-905f-4249-8912-3090d96213d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:42.735010 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631fd11d-905f-4249-8912-3090d96213d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:42.737070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631fd11d-905f-4249-8912-3090d96213d2 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:42.746825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1353/2699] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:43.969537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88293b37-319d-46b1-9e0b-2256a214b980 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:44.036426 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88293b37-319d-46b1-9e0b-2256a214b980 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:44.040976 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:44.041651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:44.041917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:44.042340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] PATH_INFO: `/v3/users/d30a97daef914ace90075f13056dc58c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:44.043492 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d30a97daef914ace90075f13056dc58c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:44.045420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:44.234032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd30a97daef914ace90075f13056dc58c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:44.234610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd30a97daef914ace90075f13056dc58c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:44.235217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd30a97daef914ace90075f13056dc58c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:44.349468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd30a97daef914ace90075f13056dc58c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:44.357416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd30a97daef914ace90075f13056dc58c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:44.376410 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1347/2700] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:43 2022] DELETE /identity/v3/users/d30a97daef914ace90075f13056dc58c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:44.381333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779b61-8733-419e-9e0e-a49153205b29 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:44.381829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779b61-8733-419e-9e0e-a49153205b29 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:44.381938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779b61-8733-419e-9e0e-a49153205b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:44.401383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-15779b61-8733-419e-9e0e-a49153205b29 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:44.402654 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-15779b61-8733-419e-9e0e-a49153205b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:44.445969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:44.463939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1354/2701] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:44.474882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:44.476327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:44.538651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250947d2-600b-4bd4-bb98-a701bbb629f1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:44.542833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703353d-2ec7-421d-ba83-1500bda67977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:44.543521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703353d-2ec7-421d-ba83-1500bda67977 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:44.543861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703353d-2ec7-421d-ba83-1500bda67977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:44.544303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703353d-2ec7-421d-ba83-1500bda67977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:44.567142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703353d-2ec7-421d-ba83-1500bda67977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:44.569158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703353d-2ec7-421d-ba83-1500bda67977 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:44.589150 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1348/2702] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:45.379990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15779b61-8733-419e-9e0e-a49153205b29 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:45.381462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-15779b61-8733-419e-9e0e-a49153205b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:45.394298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15779b61-8733-419e-9e0e-a49153205b29 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:45.399486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:45.407047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:45.407047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:45.407047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] PATH_INFO: `/v3/projects/eb9aacaa61a548eda727d96ea34f0786` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:45.407047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9aacaa61a548eda727d96ea34f0786)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:45.407047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:45.547019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9aacaa61a548eda727d96ea34f0786', 'request_id': 'req-840a0267-9717-474b-b9b4-7aa2d76f010d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:45.553509 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9aacaa61a548eda727d96ea34f0786', 'request_id': 'req-840a0267-9717-474b-b9b4-7aa2d76f010d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:45.641768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9aacaa61a548eda727d96ea34f0786', 'request_id': 'req-840a0267-9717-474b-b9b4-7aa2d76f010d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:45.654797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] The token cache is being invalidate because project eb9aacaa61a548eda727d96ea34f0786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:45.655142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:45.658524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:45.658524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1355/2703] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:45 2022] DELETE /identity/v3/projects/eb9aacaa61a548eda727d96ea34f0786 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:45.662014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703353d-2ec7-421d-ba83-1500bda67977 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:45.667059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:45.672631 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703353d-2ec7-421d-ba83-1500bda67977 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:45.733134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:45.733764 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:45.734546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:45.734546 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] PATH_INFO: `/v3/users/ca11c16215c84181bdb2e85ed40a2fc9` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:45.735504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca11c16215c84181bdb2e85ed40a2fc9)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:45.738004 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:45.963544 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca11c16215c84181bdb2e85ed40a2fc9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:45.964207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca11c16215c84181bdb2e85ed40a2fc9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:45.964935 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca11c16215c84181bdb2e85ed40a2fc9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:46.057966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca11c16215c84181bdb2e85ed40a2fc9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:46.066355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0735224a-589f-41cb-9b01-9a6d1c0b2973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca11c16215c84181bdb2e85ed40a2fc9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:46.084957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1349/2704] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:45 2022] DELETE /identity/v3/users/ca11c16215c84181bdb2e85ed40a2fc9 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:46.671732 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:46.673724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:46.688774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a0267-9717-474b-b9b4-7aa2d76f010d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:46.752182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:46.752821 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:46.753108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:46.753386 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] PATH_INFO: `/v3/projects/27057e8771f746f0bc963a55f28120e6` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:46.754558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=27057e8771f746f0bc963a55f28120e6)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:46.756750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:46.905219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27057e8771f746f0bc963a55f28120e6', 'request_id': 'req-afb5945c-cff0-454a-88d6-0b0805c63a1c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:46.911269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27057e8771f746f0bc963a55f28120e6', 'request_id': 'req-afb5945c-cff0-454a-88d6-0b0805c63a1c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:47.023803 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27057e8771f746f0bc963a55f28120e6', 'request_id': 'req-afb5945c-cff0-454a-88d6-0b0805c63a1c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:47.032120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:47.041564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] The token cache is being invalidate because project 27057e8771f746f0bc963a55f28120e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:47.041564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:47.041564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-afb5945c-cff0-454a-88d6-0b0805c63a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:47.041564 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1356/2705] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:46 2022] DELETE /identity/v3/projects/27057e8771f746f0bc963a55f28120e6 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:47.044014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056daf1-05df-4b89-975b-0f08325abed1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.044428 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056daf1-05df-4b89-975b-0f08325abed1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.044574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056daf1-05df-4b89-975b-0f08325abed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.066813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3056daf1-05df-4b89-975b-0f08325abed1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:47.068068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3056daf1-05df-4b89-975b-0f08325abed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:47.133814 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1350/2706] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:47.142547 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b581ae-e81b-4d2f-83d4-d25d82faf355 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.142832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b581ae-e81b-4d2f-83d4-d25d82faf355 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.143106 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b581ae-e81b-4d2f-83d4-d25d82faf355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.163756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-43b581ae-e81b-4d2f-83d4-d25d82faf355 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:47.165042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-43b581ae-e81b-4d2f-83d4-d25d82faf355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:47.174283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1357/2707] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:47.179733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056daf1-05df-4b89-975b-0f08325abed1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:47.191087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056daf1-05df-4b89-975b-0f08325abed1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:47.200447 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c44280-af7b-4db3-966e-fa995f667689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:47.201058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c44280-af7b-4db3-966e-fa995f667689 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.201335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c44280-af7b-4db3-966e-fa995f667689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.201603 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c44280-af7b-4db3-966e-fa995f667689 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.202306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c44280-af7b-4db3-966e-fa995f667689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:37:47.202608 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c44280-af7b-4db3-966e-fa995f667689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:47.204233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c44280-af7b-4db3-966e-fa995f667689 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:47.217163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1351/2708] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:37:47.222121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b581ae-e81b-4d2f-83d4-d25d82faf355 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:47.231449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b581ae-e81b-4d2f-83d4-d25d82faf355 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:47.236394 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623a16b-c797-40be-8de4-1d93b6b07e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:47.237149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623a16b-c797-40be-8de4-1d93b6b07e20 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.237271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623a16b-c797-40be-8de4-1d93b6b07e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.237525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623a16b-c797-40be-8de4-1d93b6b07e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.238219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623a16b-c797-40be-8de4-1d93b6b07e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:47.240104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623a16b-c797-40be-8de4-1d93b6b07e20 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:47.323213 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1358/2709] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:47 2022] POST /identity/v3/projects => generated 359 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:47.327343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c44280-af7b-4db3-966e-fa995f667689 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:47.338129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c44280-af7b-4db3-966e-fa995f667689 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:47.343072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:47.397713 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d259edc-a4b2-4ec9-9357-3fbe5c8cdfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:47.398343 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d259edc-a4b2-4ec9-9357-3fbe5c8cdfad admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.398619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d259edc-a4b2-4ec9-9357-3fbe5c8cdfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.398890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d259edc-a4b2-4ec9-9357-3fbe5c8cdfad admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.399610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d259edc-a4b2-4ec9-9357-3fbe5c8cdfad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:47.402207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d259edc-a4b2-4ec9-9357-3fbe5c8cdfad admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:47.514533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1352/2710] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:47 2022] POST /identity/v3/users => generated 280 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:47.519274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623a16b-c797-40be-8de4-1d93b6b07e20 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:47.530095 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623a16b-c797-40be-8de4-1d93b6b07e20 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:47.535205 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75cd9f-944d-4257-9225-e0936bae47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:47.535799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75cd9f-944d-4257-9225-e0936bae47cd admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.536081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75cd9f-944d-4257-9225-e0936bae47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.536353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75cd9f-944d-4257-9225-e0936bae47cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.537059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75cd9f-944d-4257-9225-e0936bae47cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:47.538691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75cd9f-944d-4257-9225-e0936bae47cd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:47.549298 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1359/2711] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:47.553571 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d259edc-a4b2-4ec9-9357-3fbe5c8cdfad admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:47.563467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d259edc-a4b2-4ec9-9357-3fbe5c8cdfad admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:47.568103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c36424-d404-4403-9dbd-7e9fa40a2ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:47.568699 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c36424-d404-4403-9dbd-7e9fa40a2ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.568980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c36424-d404-4403-9dbd-7e9fa40a2ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.569256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c36424-d404-4403-9dbd-7e9fa40a2ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.569942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c36424-d404-4403-9dbd-7e9fa40a2ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:47.571654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c36424-d404-4403-9dbd-7e9fa40a2ba7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:47.583118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1353/2712] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:47.589774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75cd9f-944d-4257-9225-e0936bae47cd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:47.600867 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75cd9f-944d-4257-9225-e0936bae47cd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:47.605717 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1115b0-b523-4b89-a765-97b82855a034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:47.606406 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1115b0-b523-4b89-a765-97b82855a034 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.606684 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1115b0-b523-4b89-a765-97b82855a034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.606983 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1115b0-b523-4b89-a765-97b82855a034 admin admin] PATH_INFO: `/v3/projects/6763335d5df040e7b248dd271cd35f05/users/22282403e8314f2ba305a0485d776db3/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.622662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1115b0-b523-4b89-a765-97b82855a034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6763335d5df040e7b248dd271cd35f05, user_id=22282403e8314f2ba305a0485d776db3, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:47.625334 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1115b0-b523-4b89-a765-97b82855a034 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:47.773366 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1360/2713] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:47 2022] PUT /identity/v3/projects/6763335d5df040e7b248dd271cd35f05/users/22282403e8314f2ba305a0485d776db3/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:37:47.783836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb6ac5-6df7-4027-b4d4-0344cc3fb667 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.784049 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb6ac5-6df7-4027-b4d4-0344cc3fb667 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.784338 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb6ac5-6df7-4027-b4d4-0344cc3fb667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.812217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a6eb6ac5-6df7-4027-b4d4-0344cc3fb667 None None] MFA Rules not processed for user `22282403e8314f2ba305a0485d776db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:47.813504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a6eb6ac5-6df7-4027-b4d4-0344cc3fb667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:47.881841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1354/2714] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:47.917605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1115b0-b523-4b89-a765-97b82855a034 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:47.919397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5a1115b0-b523-4b89-a765-97b82855a034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:47.977186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1115b0-b523-4b89-a765-97b82855a034 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:47.980943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49999e8-1063-425c-8229-a03938d3d194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:47.981552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49999e8-1063-425c-8229-a03938d3d194 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.981834 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49999e8-1063-425c-8229-a03938d3d194 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.982176 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49999e8-1063-425c-8229-a03938d3d194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.991356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49999e8-1063-425c-8229-a03938d3d194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:47.992797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49999e8-1063-425c-8229-a03938d3d194 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:48.002487 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1361/2715] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:48.017680 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb6ac5-6df7-4027-b4d4-0344cc3fb667 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:48.078362 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb6ac5-6df7-4027-b4d4-0344cc3fb667 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:48.083747 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:48.084406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:48.084690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:48.084963 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] PATH_INFO: `/v3/users/22282403e8314f2ba305a0485d776db3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:48.086135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=22282403e8314f2ba305a0485d776db3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:48.088169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:48.109420 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:48.420405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22282403e8314f2ba305a0485d776db3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.421155 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22282403e8314f2ba305a0485d776db3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.421717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22282403e8314f2ba305a0485d776db3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.513939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22282403e8314f2ba305a0485d776db3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.521844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22282403e8314f2ba305a0485d776db3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.541543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1355/2716] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:48 2022] DELETE /identity/v3/users/22282403e8314f2ba305a0485d776db3 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:48.546136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aebdb-8be1-4a14-8309-701261f46cae None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:48.546428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aebdb-8be1-4a14-8309-701261f46cae None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:48.546703 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aebdb-8be1-4a14-8309-701261f46cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:48.566356 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-209aebdb-8be1-4a14-8309-701261f46cae None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:48.567585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-209aebdb-8be1-4a14-8309-701261f46cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:48.627605 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1362/2717] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:48.637656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:48.639158 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:48.696922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245f207-1580-448f-b7a9-8c1222b874cf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:48.700590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d275c-129c-4980-9752-6123ec1aa848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:48.701189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d275c-129c-4980-9752-6123ec1aa848 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:48.701470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d275c-129c-4980-9752-6123ec1aa848 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:48.701742 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d275c-129c-4980-9752-6123ec1aa848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:48.711555 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d275c-129c-4980-9752-6123ec1aa848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:48.713066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d275c-129c-4980-9752-6123ec1aa848 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:48.723559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1356/2718] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:49.545768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aebdb-8be1-4a14-8309-701261f46cae None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:49.545768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-209aebdb-8be1-4a14-8309-701261f46cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:49.556459 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aebdb-8be1-4a14-8309-701261f46cae None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:49.561678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:49.562522 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:49.562775 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:49.563050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] PATH_INFO: `/v3/projects/6763335d5df040e7b248dd271cd35f05` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:49.564705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6763335d5df040e7b248dd271cd35f05)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:49.567548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:49.747005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:49.756569 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6763335d5df040e7b248dd271cd35f05', 'request_id': 'req-481890b9-8cc5-486b-a21f-00a52046a841'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:49.762457 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6763335d5df040e7b248dd271cd35f05', 'request_id': 'req-481890b9-8cc5-486b-a21f-00a52046a841'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:49.886261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6763335d5df040e7b248dd271cd35f05', 'request_id': 'req-481890b9-8cc5-486b-a21f-00a52046a841'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:49.899172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] The token cache is being invalidate because project 6763335d5df040e7b248dd271cd35f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:49.899728 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:49.900344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-481890b9-8cc5-486b-a21f-00a52046a841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:49.901895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1363/2719] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:49 2022] DELETE /identity/v3/projects/6763335d5df040e7b248dd271cd35f05 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:49.906717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ef2b1-d85f-403c-8544-e446dd9f656a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:49.907185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ef2b1-d85f-403c-8544-e446dd9f656a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:49.907303 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ef2b1-d85f-403c-8544-e446dd9f656a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:49.926485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-696ef2b1-d85f-403c-8544-e446dd9f656a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:49.927712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-696ef2b1-d85f-403c-8544-e446dd9f656a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:49.987841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1357/2720] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:49.995661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b7d77-064e-4df5-9fe7-4cea8738290e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.000567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b7d77-064e-4df5-9fe7-4cea8738290e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.000567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b7d77-064e-4df5-9fe7-4cea8738290e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.016990 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bb9b7d77-064e-4df5-9fe7-4cea8738290e None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:50.018229 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bb9b7d77-064e-4df5-9fe7-4cea8738290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:50.027299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1364/2721] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:50.031830 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ef2b1-d85f-403c-8544-e446dd9f656a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:50.046507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ef2b1-d85f-403c-8544-e446dd9f656a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:50.046752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85439ef0-9d6d-415b-89b8-03592aba96f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:50.047288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85439ef0-9d6d-415b-89b8-03592aba96f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.047568 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85439ef0-9d6d-415b-89b8-03592aba96f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.047844 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85439ef0-9d6d-415b-89b8-03592aba96f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.048629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85439ef0-9d6d-415b-89b8-03592aba96f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:37:50.048941 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85439ef0-9d6d-415b-89b8-03592aba96f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:50.050589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85439ef0-9d6d-415b-89b8-03592aba96f6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:50.062715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1358/2722] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:37:50.068698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b7d77-064e-4df5-9fe7-4cea8738290e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:50.078296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b7d77-064e-4df5-9fe7-4cea8738290e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:50.084152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c5930-2b97-4146-b8da-0145af18aafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:50.084763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c5930-2b97-4146-b8da-0145af18aafd admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.085137 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c5930-2b97-4146-b8da-0145af18aafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.085416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c5930-2b97-4146-b8da-0145af18aafd admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.086082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c5930-2b97-4146-b8da-0145af18aafd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:50.088041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c5930-2b97-4146-b8da-0145af18aafd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:50.208359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1365/2723] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:50 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:50.212622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85439ef0-9d6d-415b-89b8-03592aba96f6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:50.223029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85439ef0-9d6d-415b-89b8-03592aba96f6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:50.248961 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:50.282909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f929f2-08c1-4bbe-af03-73f9f04e3162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:50.283537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f929f2-08c1-4bbe-af03-73f9f04e3162 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.288016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f929f2-08c1-4bbe-af03-73f9f04e3162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.288016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f929f2-08c1-4bbe-af03-73f9f04e3162 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.288016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f929f2-08c1-4bbe-af03-73f9f04e3162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:50.288016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f929f2-08c1-4bbe-af03-73f9f04e3162 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:50.401217 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1359/2724] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:50 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:50.405587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c5930-2b97-4146-b8da-0145af18aafd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:50.416141 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c5930-2b97-4146-b8da-0145af18aafd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:50.420806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f12b3a-8af6-42e4-82ed-b6d61256c242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:50.421416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f12b3a-8af6-42e4-82ed-b6d61256c242 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.421698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f12b3a-8af6-42e4-82ed-b6d61256c242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.422013 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f12b3a-8af6-42e4-82ed-b6d61256c242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.422673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f12b3a-8af6-42e4-82ed-b6d61256c242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:50.424293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f12b3a-8af6-42e4-82ed-b6d61256c242 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:50.435789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1366/2725] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:50.440136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f929f2-08c1-4bbe-af03-73f9f04e3162 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:50.453936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f929f2-08c1-4bbe-af03-73f9f04e3162 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:50.461788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbc10e-e24c-4b3a-a286-666f10457037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:50.462441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbc10e-e24c-4b3a-a286-666f10457037 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.462726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbc10e-e24c-4b3a-a286-666f10457037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.462999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbc10e-e24c-4b3a-a286-666f10457037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.463684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbc10e-e24c-4b3a-a286-666f10457037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:50.465274 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbc10e-e24c-4b3a-a286-666f10457037 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:50.476180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1360/2726] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:50.480662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f12b3a-8af6-42e4-82ed-b6d61256c242 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:50.491470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f12b3a-8af6-42e4-82ed-b6d61256c242 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:50.496172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf3414-bdac-4bb4-8be6-53fda2c50867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:50.496822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3414-bdac-4bb4-8be6-53fda2c50867 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.497105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3414-bdac-4bb4-8be6-53fda2c50867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.497382 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3414-bdac-4bb4-8be6-53fda2c50867 admin admin] PATH_INFO: `/v3/projects/687d14b737b44b19a841424ced5aa951/users/ad02d985c3ab4ad380e8993521ed4a55/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.514193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf3414-bdac-4bb4-8be6-53fda2c50867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=687d14b737b44b19a841424ced5aa951, user_id=ad02d985c3ab4ad380e8993521ed4a55, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:50.516989 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf3414-bdac-4bb4-8be6-53fda2c50867 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:50.659069 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1367/2727] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:50 2022] PUT /identity/v3/projects/687d14b737b44b19a841424ced5aa951/users/ad02d985c3ab4ad380e8993521ed4a55/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:50.729769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe203bf-2e64-4ce1-8254-39bf46040251 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.730099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe203bf-2e64-4ce1-8254-39bf46040251 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.730408 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe203bf-2e64-4ce1-8254-39bf46040251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.758445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2fe203bf-2e64-4ce1-8254-39bf46040251 None None] MFA Rules not processed for user `ad02d985c3ab4ad380e8993521ed4a55`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:50.759777 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2fe203bf-2e64-4ce1-8254-39bf46040251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:50.829335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1361/2728] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:50.834399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf3414-bdac-4bb4-8be6-53fda2c50867 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:50.899817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf3414-bdac-4bb4-8be6-53fda2c50867 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:50.904650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c8f44-f7e3-4570-ae7d-22ff8ff97dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:50.905188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c8f44-f7e3-4570-ae7d-22ff8ff97dcd admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.905467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c8f44-f7e3-4570-ae7d-22ff8ff97dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.905744 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c8f44-f7e3-4570-ae7d-22ff8ff97dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.906496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c8f44-f7e3-4570-ae7d-22ff8ff97dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:50.908669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c8f44-f7e3-4570-ae7d-22ff8ff97dcd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:51.028854 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1368/2729] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:50 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:51.030349 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe203bf-2e64-4ce1-8254-39bf46040251 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:51.041276 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe203bf-2e64-4ce1-8254-39bf46040251 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:51.100344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a28d3-5b52-47a8-8e6d-fc36048a3c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:51.100951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a28d3-5b52-47a8-8e6d-fc36048a3c28 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:51.101233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a28d3-5b52-47a8-8e6d-fc36048a3c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:51.101501 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a28d3-5b52-47a8-8e6d-fc36048a3c28 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:51.102192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a28d3-5b52-47a8-8e6d-fc36048a3c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:51.104164 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a28d3-5b52-47a8-8e6d-fc36048a3c28 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:51.242087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1362/2730] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:51 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:51.246094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c8f44-f7e3-4570-ae7d-22ff8ff97dcd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:51.256526 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c8f44-f7e3-4570-ae7d-22ff8ff97dcd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:51.262441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4aaadd-12a2-4a0a-b624-d58bc7c2f442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:51.262999 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aaadd-12a2-4a0a-b624-d58bc7c2f442 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:51.263311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aaadd-12a2-4a0a-b624-d58bc7c2f442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:51.263563 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aaadd-12a2-4a0a-b624-d58bc7c2f442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:51.264222 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4aaadd-12a2-4a0a-b624-d58bc7c2f442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:51.265851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4aaadd-12a2-4a0a-b624-d58bc7c2f442 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:51.276905 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1369/2731] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:51.280948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a28d3-5b52-47a8-8e6d-fc36048a3c28 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:51.291915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a28d3-5b52-47a8-8e6d-fc36048a3c28 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:51.296648 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9024770-2fb3-426b-a208-33ef810bfd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:51.297443 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9024770-2fb3-426b-a208-33ef810bfd13 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:51.297729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9024770-2fb3-426b-a208-33ef810bfd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:51.298040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9024770-2fb3-426b-a208-33ef810bfd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:51.298698 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9024770-2fb3-426b-a208-33ef810bfd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:51.300869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9024770-2fb3-426b-a208-33ef810bfd13 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:51.312203 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1363/2732] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:51.318323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4aaadd-12a2-4a0a-b624-d58bc7c2f442 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:51.329186 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4aaadd-12a2-4a0a-b624-d58bc7c2f442 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:51.334534 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea99840-d9ed-46c1-a49b-3c080b8878f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:51.335200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea99840-d9ed-46c1-a49b-3c080b8878f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:51.335484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea99840-d9ed-46c1-a49b-3c080b8878f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:51.335760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea99840-d9ed-46c1-a49b-3c080b8878f7 admin admin] PATH_INFO: `/v3/projects/a04d6f905f6a4c858de870d370eb2159/users/874b45ba89de4d80b505cbb21f93ce2b/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:51.351157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea99840-d9ed-46c1-a49b-3c080b8878f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04d6f905f6a4c858de870d370eb2159, user_id=874b45ba89de4d80b505cbb21f93ce2b, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:51.353835 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea99840-d9ed-46c1-a49b-3c080b8878f7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:51.491918 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1370/2733] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:51 2022] PUT /identity/v3/projects/a04d6f905f6a4c858de870d370eb2159/users/874b45ba89de4d80b505cbb21f93ce2b/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:51.502266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ad525-469b-4f01-8670-39a7e1d0e2ec None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:51.513228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ad525-469b-4f01-8670-39a7e1d0e2ec None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:51.513228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ad525-469b-4f01-8670-39a7e1d0e2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:51.529224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:51.532442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-908ad525-469b-4f01-8670-39a7e1d0e2ec None None] MFA Rules not processed for user `874b45ba89de4d80b505cbb21f93ce2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:51.533678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-908ad525-469b-4f01-8670-39a7e1d0e2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:51.602524 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1364/2734] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:51.607332 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea99840-d9ed-46c1-a49b-3c080b8878f7 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:51.618822 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:51.687641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea99840-d9ed-46c1-a49b-3c080b8878f7 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:51.696402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7d8f3-e677-42eb-bb3e-f5b2407f1c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:51.696402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7d8f3-e677-42eb-bb3e-f5b2407f1c67 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:51.696402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7d8f3-e677-42eb-bb3e-f5b2407f1c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:51.696402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7d8f3-e677-42eb-bb3e-f5b2407f1c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:51.696870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7d8f3-e677-42eb-bb3e-f5b2407f1c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:51.698112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7d8f3-e677-42eb-bb3e-f5b2407f1c67 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:51.825333 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1371/2735] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:51 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:51.829831 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ad525-469b-4f01-8670-39a7e1d0e2ec None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:51.843791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ad525-469b-4f01-8670-39a7e1d0e2ec None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:51.935305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d09a2-6912-4114-95f0-cd176b96bb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:51.935857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d09a2-6912-4114-95f0-cd176b96bb4b admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:51.936341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d09a2-6912-4114-95f0-cd176b96bb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:51.936726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d09a2-6912-4114-95f0-cd176b96bb4b admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:51.937567 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d09a2-6912-4114-95f0-cd176b96bb4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:51.939655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d09a2-6912-4114-95f0-cd176b96bb4b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:52.018965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1365/2736] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:51 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:52.023006 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7d8f3-e677-42eb-bb3e-f5b2407f1c67 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:52.035245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7d8f3-e677-42eb-bb3e-f5b2407f1c67 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:52.045253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73853a-bbf3-412e-b936-12644c2bc9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:52.045253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73853a-bbf3-412e-b936-12644c2bc9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:52.045253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73853a-bbf3-412e-b936-12644c2bc9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:52.045253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73853a-bbf3-412e-b936-12644c2bc9f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:52.045704 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73853a-bbf3-412e-b936-12644c2bc9f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:52.046050 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73853a-bbf3-412e-b936-12644c2bc9f5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:52.063460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1372/2737] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:52.064743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d09a2-6912-4114-95f0-cd176b96bb4b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:52.077188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d09a2-6912-4114-95f0-cd176b96bb4b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:52.083782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fda94-6468-4ebd-8c21-3ad08e0345e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:52.084605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fda94-6468-4ebd-8c21-3ad08e0345e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:52.084892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fda94-6468-4ebd-8c21-3ad08e0345e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:52.085166 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fda94-6468-4ebd-8c21-3ad08e0345e2 admin admin] PATH_INFO: `/v3/projects/6cbff441a27d457889b20bbe2b6f42cd/users/69253b321f0a4636b9d26f3822532625/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:52.103225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fda94-6468-4ebd-8c21-3ad08e0345e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cbff441a27d457889b20bbe2b6f42cd, user_id=69253b321f0a4636b9d26f3822532625, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:52.106344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fda94-6468-4ebd-8c21-3ad08e0345e2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:52.208991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1366/2738] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:52 2022] PUT /identity/v3/projects/6cbff441a27d457889b20bbe2b6f42cd/users/69253b321f0a4636b9d26f3822532625/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:52.213477 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73853a-bbf3-412e-b936-12644c2bc9f5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:52.226415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73853a-bbf3-412e-b936-12644c2bc9f5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:52.290073 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a1175-1af2-4920-9451-2c96795cce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:52.290699 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a1175-1af2-4920-9451-2c96795cce40 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:52.290979 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a1175-1af2-4920-9451-2c96795cce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:52.291254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a1175-1af2-4920-9451-2c96795cce40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:52.292284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a1175-1af2-4920-9451-2c96795cce40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:52.293948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a1175-1af2-4920-9451-2c96795cce40 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:52.301115 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fda94-6468-4ebd-8c21-3ad08e0345e2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:52.304161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6b3fda94-6468-4ebd-8c21-3ad08e0345e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:52.312677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1373/2739] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:52.317629 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a1175-1af2-4920-9451-2c96795cce40 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:52.328587 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a1175-1af2-4920-9451-2c96795cce40 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:52.334694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54036099-9bfd-401f-9dc4-9dd68e998566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:52.335463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54036099-9bfd-401f-9dc4-9dd68e998566 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:52.335742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54036099-9bfd-401f-9dc4-9dd68e998566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:52.336033 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54036099-9bfd-401f-9dc4-9dd68e998566 admin admin] PATH_INFO: `/v3/projects/6cbff441a27d457889b20bbe2b6f42cd/users/69253b321f0a4636b9d26f3822532625/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:52.338215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54036099-9bfd-401f-9dc4-9dd68e998566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cbff441a27d457889b20bbe2b6f42cd, user_id=69253b321f0a4636b9d26f3822532625, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:52.340917 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54036099-9bfd-401f-9dc4-9dd68e998566 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:52.357226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1374/2740] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:52 2022] PUT /identity/v3/projects/6cbff441a27d457889b20bbe2b6f42cd/users/69253b321f0a4636b9d26f3822532625/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:37:52.368239 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12cb8a-769a-41bf-9341-da1bb98f7756 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:52.368519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12cb8a-769a-41bf-9341-da1bb98f7756 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:52.368793 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12cb8a-769a-41bf-9341-da1bb98f7756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:52.400813 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3c12cb8a-769a-41bf-9341-da1bb98f7756 None None] MFA Rules not processed for user `69253b321f0a4636b9d26f3822532625`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:52.402132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3c12cb8a-769a-41bf-9341-da1bb98f7756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:52.410547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fda94-6468-4ebd-8c21-3ad08e0345e2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:52.473000 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:52.509179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1375/2741] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:37:52.521820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12cb8a-769a-41bf-9341-da1bb98f7756 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:52.529745 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3c12cb8a-769a-41bf-9341-da1bb98f7756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:52.559911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd85997-6978-4d2d-a1f6-2ad25ae9227b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:52.566156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd85997-6978-4d2d-a1f6-2ad25ae9227b service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:52.566156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd85997-6978-4d2d-a1f6-2ad25ae9227b service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:52.566156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd85997-6978-4d2d-a1f6-2ad25ae9227b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:52.569853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dfd85997-6978-4d2d-a1f6-2ad25ae9227b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:52.588075 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd85997-6978-4d2d-a1f6-2ad25ae9227b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:52.589795 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd85997-6978-4d2d-a1f6-2ad25ae9227b service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:52.612965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1367/2742] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:52.627179 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12cb8a-769a-41bf-9341-da1bb98f7756 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:52.632407 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03871c70-2876-4196-929d-db47df24e8fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:52.633627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03871c70-2876-4196-929d-db47df24e8fc service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:52.634762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03871c70-2876-4196-929d-db47df24e8fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:52.635136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03871c70-2876-4196-929d-db47df24e8fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:52.705214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03871c70-2876-4196-929d-db47df24e8fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:52.706814 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03871c70-2876-4196-929d-db47df24e8fc service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:52.717367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1376/2743] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:53.572631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:56.165581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd85997-6978-4d2d-a1f6-2ad25ae9227b service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:56.175025 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd85997-6978-4d2d-a1f6-2ad25ae9227b service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:56.179325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca9646-a16b-4a0f-9dbc-eb55f72739a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:56.179927 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca9646-a16b-4a0f-9dbc-eb55f72739a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:56.180201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca9646-a16b-4a0f-9dbc-eb55f72739a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:56.180471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca9646-a16b-4a0f-9dbc-eb55f72739a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:56.239223 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca9646-a16b-4a0f-9dbc-eb55f72739a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:56.240650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca9646-a16b-4a0f-9dbc-eb55f72739a1 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:56.249851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1368/2744] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:57.427038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731339a-1021-4f23-89eb-1af1fe7b9bde None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:57.427538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731339a-1021-4f23-89eb-1af1fe7b9bde None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:57.427636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731339a-1021-4f23-89eb-1af1fe7b9bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:57.446845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1731339a-1021-4f23-89eb-1af1fe7b9bde None None] MFA Rules not processed for user `4dfbe3078d714129b5a6e6e45dbc0b25`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:57.448048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1731339a-1021-4f23-89eb-1af1fe7b9bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:57.509467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1377/2745] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:37:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:57.839283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca9646-a16b-4a0f-9dbc-eb55f72739a1 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:57.840896 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f4ca9646-a16b-4a0f-9dbc-eb55f72739a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:57.901735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca9646-a16b-4a0f-9dbc-eb55f72739a1 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:57.905316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:57.905990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:57.906266 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:57.906642 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:57.908761 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:57.922293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:57.924029 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:57.933015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1369/2746] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:58.045643 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731339a-1021-4f23-89eb-1af1fe7b9bde None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:58.055492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731339a-1021-4f23-89eb-1af1fe7b9bde None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:58.059062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a00a2-666d-4829-b2ca-e087a51012df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:58.059670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a00a2-666d-4829-b2ca-e087a51012df service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:58.059964 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a00a2-666d-4829-b2ca-e087a51012df service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:58.060215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a00a2-666d-4829-b2ca-e087a51012df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:58.072305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a00a2-666d-4829-b2ca-e087a51012df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:58.073710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a00a2-666d-4829-b2ca-e087a51012df service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:58.084404 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1378/2747] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:02.694860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:02.696425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:02.714055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6fddd-dd94-4be3-acaa-8a96f09dcb82 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:02.720930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5481d2-0471-4b03-8989-39c2e236ebc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:02.721731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5481d2-0471-4b03-8989-39c2e236ebc2 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:02.722054 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5481d2-0471-4b03-8989-39c2e236ebc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:02.722328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5481d2-0471-4b03-8989-39c2e236ebc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:02.724129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6b5481d2-0471-4b03-8989-39c2e236ebc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:02.742037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5481d2-0471-4b03-8989-39c2e236ebc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:02.743022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5481d2-0471-4b03-8989-39c2e236ebc2 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:02.755207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1370/2748] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:04.214538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a00a2-666d-4829-b2ca-e087a51012df service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:04.259058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:04.282865 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a00a2-666d-4829-b2ca-e087a51012df service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:04.287902 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:04.288544 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:04.288830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:04.289106 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] PATH_INFO: `/v3/users/ad02d985c3ab4ad380e8993521ed4a55` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:04.290425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad02d985c3ab4ad380e8993521ed4a55)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:04.292742 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:04.581336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad02d985c3ab4ad380e8993521ed4a55'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:04.582035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad02d985c3ab4ad380e8993521ed4a55'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:04.582660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad02d985c3ab4ad380e8993521ed4a55'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:04.590231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad02d985c3ab4ad380e8993521ed4a55'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:04.603518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad02d985c3ab4ad380e8993521ed4a55'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:04.697552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1379/2749] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:38:04 2022] DELETE /identity/v3/users/ad02d985c3ab4ad380e8993521ed4a55 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:38:04.702079 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78353286-a778-4dec-993e-a21a25dbcba1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:04.702547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78353286-a778-4dec-993e-a21a25dbcba1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:04.702705 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78353286-a778-4dec-993e-a21a25dbcba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:04.723418 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-78353286-a778-4dec-993e-a21a25dbcba1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:38:04.724667 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-78353286-a778-4dec-993e-a21a25dbcba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:04.739388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:04.787230 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1371/2750] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:38:04.796955 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:04.798727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:04.857595 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344175-f7fe-464c-a261-66c5f5615f80 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:04.861173 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4352488-6d81-4646-b5a9-42ab22493dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:04.861782 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4352488-6d81-4646-b5a9-42ab22493dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:04.862090 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4352488-6d81-4646-b5a9-42ab22493dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:04.862364 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4352488-6d81-4646-b5a9-42ab22493dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:04.871882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4352488-6d81-4646-b5a9-42ab22493dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:04.873316 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4352488-6d81-4646-b5a9-42ab22493dd8 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:04.883104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1380/2751] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:05.362181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78353286-a778-4dec-993e-a21a25dbcba1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:05.363737 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-78353286-a778-4dec-993e-a21a25dbcba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:05.376659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78353286-a778-4dec-993e-a21a25dbcba1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:05.381316 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:05.382064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:05.382340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:05.382610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] PATH_INFO: `/v3/projects/687d14b737b44b19a841424ced5aa951` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:05.384516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=687d14b737b44b19a841424ced5aa951)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:05.386199 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:05.515957 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687d14b737b44b19a841424ced5aa951', 'request_id': 'req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:05.522377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687d14b737b44b19a841424ced5aa951', 'request_id': 'req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:05.528730 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687d14b737b44b19a841424ced5aa951', 'request_id': 'req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:05.605295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] The token cache is being invalidate because project 687d14b737b44b19a841424ced5aa951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:38:05.605574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:05.606228 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:05.607703 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1372/2752] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:38:05 2022] DELETE /identity/v3/projects/687d14b737b44b19a841424ced5aa951 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:38:05.611751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4352488-6d81-4646-b5a9-42ab22493dd8 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:05.621920 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4352488-6d81-4646-b5a9-42ab22493dd8 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:05.681472 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:05.682036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:05.682320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:05.682612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] PATH_INFO: `/v3/users/874b45ba89de4d80b505cbb21f93ce2b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:05.683758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] RBAC: Authorizing `identity:delete_user(user_id=874b45ba89de4d80b505cbb21f93ce2b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:05.685700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:05.687334 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:05.963524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874b45ba89de4d80b505cbb21f93ce2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:05.965627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874b45ba89de4d80b505cbb21f93ce2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:05.965627 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874b45ba89de4d80b505cbb21f93ce2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:05.972085 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874b45ba89de4d80b505cbb21f93ce2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:05.984533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874b45ba89de4d80b505cbb21f93ce2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:06.090889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1381/2753] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:38:05 2022] DELETE /identity/v3/users/874b45ba89de4d80b505cbb21f93ce2b => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:38:06.626508 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:06.627684 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:06.646141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9344fa-bfaa-4d08-b91f-52c7f78d9912 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:06.713422 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:06.714145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:06.714406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:06.714672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] PATH_INFO: `/v3/projects/a04d6f905f6a4c858de870d370eb2159` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:06.715792 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a04d6f905f6a4c858de870d370eb2159)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:06.717702 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:06.873586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:06.876620 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a04d6f905f6a4c858de870d370eb2159', 'request_id': 'req-95755d5b-89ed-41b1-972a-75d520690e73'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:06.883966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a04d6f905f6a4c858de870d370eb2159', 'request_id': 'req-95755d5b-89ed-41b1-972a-75d520690e73'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:06.890287 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a04d6f905f6a4c858de870d370eb2159', 'request_id': 'req-95755d5b-89ed-41b1-972a-75d520690e73'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:07.040575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] The token cache is being invalidate because project a04d6f905f6a4c858de870d370eb2159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:38:07.040869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:07.041531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:07.043056 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1373/2754] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:38:06 2022] DELETE /identity/v3/projects/a04d6f905f6a4c858de870d370eb2159 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:38:07.047136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:07.048598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:07.062991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfee3e-64ec-404f-9d1c-6163da2af13e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:07.122707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:07.123354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:07.123689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:07.123912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] PATH_INFO: `/v3/users/69253b321f0a4636b9d26f3822532625` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:07.125048 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=69253b321f0a4636b9d26f3822532625)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:07.127037 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:07.352030 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69253b321f0a4636b9d26f3822532625'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:07.352790 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69253b321f0a4636b9d26f3822532625'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:07.353405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69253b321f0a4636b9d26f3822532625'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:07.360500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69253b321f0a4636b9d26f3822532625'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:07.373470 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69253b321f0a4636b9d26f3822532625'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:07.498425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:07.503816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1382/2755] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:38:07 2022] DELETE /identity/v3/users/69253b321f0a4636b9d26f3822532625 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:38:08.104651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:08.106305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:08.122644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95755d5b-89ed-41b1-972a-75d520690e73 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:08.186294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:08.186928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:08.187207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:08.187478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] PATH_INFO: `/v3/projects/6cbff441a27d457889b20bbe2b6f42cd` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:08.188625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cbff441a27d457889b20bbe2b6f42cd)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:08.190580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:08.339141 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cbff441a27d457889b20bbe2b6f42cd', 'request_id': 'req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:08.344977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cbff441a27d457889b20bbe2b6f42cd', 'request_id': 'req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:08.351767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cbff441a27d457889b20bbe2b6f42cd', 'request_id': 'req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:08.491899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] The token cache is being invalidate because project 6cbff441a27d457889b20bbe2b6f42cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:38:08.492189 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:08.495386 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e5dd3b2e-9c11-4fe2-ba47-64a196a3d2c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:38:08.497007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1374/2756] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:38:08 2022] DELETE /identity/v3/projects/6cbff441a27d457889b20bbe2b6f42cd => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:38:17.731830 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:18.488073 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=11,keystone:SELECT=58,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:56.100996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:56.106724 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:56.172617 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aef65-c2d6-44c0-a8ec-86d2ef207a0e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:56.233402 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:56.234047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:56.234346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:56.234612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:56.236450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:56.249369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:56.251079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:56.262240 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1383/2757] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:39:06.257491 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=58 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:39:09.947752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448c0c9-b823-4f09-bb0c-202103ea7d87 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:39:09.948306 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448c0c9-b823-4f09-bb0c-202103ea7d87 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:39:09.948432 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448c0c9-b823-4f09-bb0c-202103ea7d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:39:09.967225 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f448c0c9-b823-4f09-bb0c-202103ea7d87 None None] MFA Rules not processed for user `2e40e7790fe94c388cb206aab7c94734`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:39:09.968982 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f448c0c9-b823-4f09-bb0c-202103ea7d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:39:10.022587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1375/2758] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:39:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:39:20.015834 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=25 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:39:52.045670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:39:52.047397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:39:52.104440 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4052a-02e1-4938-9e8c-302ae7bdff63 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:39:52.108718 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:39:52.109327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:39:52.109615 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:39:52.109909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:39:52.111673 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:39:52.182893 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:39:52.184354 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:39:52.195956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1384/2759] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:39:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:39:56.141211 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448c0c9-b823-4f09-bb0c-202103ea7d87 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:39:56.142706 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f448c0c9-b823-4f09-bb0c-202103ea7d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:39:56.198821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448c0c9-b823-4f09-bb0c-202103ea7d87 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:39:56.202936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad2dbb-7bb9-457a-8818-09fcaf6d248f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:39:56.203531 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad2dbb-7bb9-457a-8818-09fcaf6d248f service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:39:56.203813 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad2dbb-7bb9-457a-8818-09fcaf6d248f service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:39:56.204088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad2dbb-7bb9-457a-8818-09fcaf6d248f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:39:56.205808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-13ad2dbb-7bb9-457a-8818-09fcaf6d248f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:39:56.219923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad2dbb-7bb9-457a-8818-09fcaf6d248f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:39:56.221392 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad2dbb-7bb9-457a-8818-09fcaf6d248f service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:39:56.232978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1376/2760] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:39:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:40:02.191081 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=54 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:04.450873 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:04.457794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:04.466142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f24df-1799-41ca-bc0e-f9d8cd524902 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:04.471789 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:04.472379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:04.472664 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:04.472942 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:04.474784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:04.537425 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:04.539094 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:04.550257 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1385/2761] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:40:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:40:06.227490 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=26 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:14.544346 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=28 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:30.996381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d241838-5490-4101-bde5-156fa90258f5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:30.996857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d241838-5490-4101-bde5-156fa90258f5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:30.997210 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d241838-5490-4101-bde5-156fa90258f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:31.002132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:31.002132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:31.016933 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b14054-b7b6-464b-ab38-7e283330db88 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:31.020013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2d241838-5490-4101-bde5-156fa90258f5 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:40:31.021514 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2d241838-5490-4101-bde5-156fa90258f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:31.022490 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:31.023311 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:31.023770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:31.024219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:31.026197 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:31.041078 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:31.043949 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:31.057936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1386/2762] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:40:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:40:31.088214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1377/2763] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:40:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:40:31.209097 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:31.210561 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:31.281259 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d490e14-6bad-40a5-a79c-efb2052ea39b service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:31.283064 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d241838-5490-4101-bde5-156fa90258f5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:31.284613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2d241838-5490-4101-bde5-156fa90258f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:31.288558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:31.289219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:31.289498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:31.289769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] PATH_INFO: `/v3/users/2e40e7790fe94c388cb206aab7c94734` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:31.291021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e40e7790fe94c388cb206aab7c94734)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:31.293220 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:31.359939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d241838-5490-4101-bde5-156fa90258f5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:31.365456 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a58403-503b-44c4-9e18-45ea3800b8b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:31.366154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a58403-503b-44c4-9e18-45ea3800b8b8 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:31.366484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a58403-503b-44c4-9e18-45ea3800b8b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:31.366917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a58403-503b-44c4-9e18-45ea3800b8b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:31.375867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a58403-503b-44c4-9e18-45ea3800b8b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:31.378027 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a58403-503b-44c4-9e18-45ea3800b8b8 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:31.386827 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1378/2764] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:40:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 11:40:31.479756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e40e7790fe94c388cb206aab7c94734'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:31.480344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e40e7790fe94c388cb206aab7c94734'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:31.480948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e40e7790fe94c388cb206aab7c94734'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:31.489755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e40e7790fe94c388cb206aab7c94734'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:31.509705 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3848a734-ef15-425d-ae79-e15000249909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e40e7790fe94c388cb206aab7c94734'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:31.591578 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1387/2765] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:40:31 2022] DELETE /identity/v3/users/2e40e7790fe94c388cb206aab7c94734 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:40:31.879326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a58403-503b-44c4-9e18-45ea3800b8b8 service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:31.895854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a58403-503b-44c4-9e18-45ea3800b8b8 service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:31.978301 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:31.978940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:31.979224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:31.979506 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] PATH_INFO: `/v3/projects/3583d4d34a634ea9a30c1f96ba70a664` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:31.980837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3583d4d34a634ea9a30c1f96ba70a664)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:31.982726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:32.132991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:32.143613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3583d4d34a634ea9a30c1f96ba70a664', 'request_id': 'req-04d13b5a-3887-4ded-9027-935f6267f1ac'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:32.150040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3583d4d34a634ea9a30c1f96ba70a664', 'request_id': 'req-04d13b5a-3887-4ded-9027-935f6267f1ac'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:32.157791 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3583d4d34a634ea9a30c1f96ba70a664', 'request_id': 'req-04d13b5a-3887-4ded-9027-935f6267f1ac'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:32.266240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] The token cache is being invalidate because project 3583d4d34a634ea9a30c1f96ba70a664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:40:32.266783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:32.267962 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-04d13b5a-3887-4ded-9027-935f6267f1ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:40:32.270562 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1379/2766] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:40:31 2022] DELETE /identity/v3/projects/3583d4d34a634ea9a30c1f96ba70a664 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:40:41.587361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:42.268173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:49.490231 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053109b-4890-40a1-b8b8-e96c7f7a0742 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:49.492323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053109b-4890-40a1-b8b8-e96c7f7a0742 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:49.492323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053109b-4890-40a1-b8b8-e96c7f7a0742 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:49.492520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1388/2767] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:40:49.498400 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684ada4-8ed1-4e3b-86b3-52f880e13ba0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:49.498940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684ada4-8ed1-4e3b-86b3-52f880e13ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:49.499032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684ada4-8ed1-4e3b-86b3-52f880e13ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:49.520800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9684ada4-8ed1-4e3b-86b3-52f880e13ba0 None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:40:49.521324 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9684ada4-8ed1-4e3b-86b3-52f880e13ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:49.590215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1380/2768] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:40:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:40:49.601721 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053109b-4890-40a1-b8b8-e96c7f7a0742 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:49.603448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2053109b-4890-40a1-b8b8-e96c7f7a0742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:49.684005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053109b-4890-40a1-b8b8-e96c7f7a0742 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:49.687669 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8aa64-1262-4748-b252-c3f848389a82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f214c2b77d4cbab2f15122ba21bf38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:49.688272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8aa64-1262-4748-b252-c3f848389a82 service glance] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:49.688580 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8aa64-1262-4748-b252-c3f848389a82 service glance] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:49.688859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8aa64-1262-4748-b252-c3f848389a82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:49.701161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8aa64-1262-4748-b252-c3f848389a82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:49.702640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8aa64-1262-4748-b252-c3f848389a82 service glance] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:49.714060 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1389/2769] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:40:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:40:49.933248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64852f6e-fea4-485d-9600-241089163153 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:49.933465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64852f6e-fea4-485d-9600-241089163153 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:49.933731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64852f6e-fea4-485d-9600-241089163153 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:49.935119 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1381/2770] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:40:49.939883 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582ca8-e6b4-473e-86bf-24098e689f1e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:49.940172 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582ca8-e6b4-473e-86bf-24098e689f1e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:49.940451 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582ca8-e6b4-473e-86bf-24098e689f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:49.980226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0f582ca8-e6b4-473e-86bf-24098e689f1e None None] MFA Rules not processed for user `b0145242ef784168bada8348ccd95f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:40:49.981498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0f582ca8-e6b4-473e-86bf-24098e689f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:50.156988 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1390/2771] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:40:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:40:50.169398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64852f6e-fea4-485d-9600-241089163153 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:50.171222 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-64852f6e-fea4-485d-9600-241089163153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:50.185219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64852f6e-fea4-485d-9600-241089163153 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:50.188733 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55cd81-8e5a-45e4-a7f0-1c3ce3cac5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:50.189323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55cd81-8e5a-45e4-a7f0-1c3ce3cac5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:50.189595 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55cd81-8e5a-45e4-a7f0-1c3ce3cac5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:50.189864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55cd81-8e5a-45e4-a7f0-1c3ce3cac5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:50.198124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55cd81-8e5a-45e4-a7f0-1c3ce3cac5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:50.199573 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55cd81-8e5a-45e4-a7f0-1c3ce3cac5e8 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:50.207474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1382/2772] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:40:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:40:50.292373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cfab0-87f3-4778-80ea-4e5a844220f1 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:50.296555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cfab0-87f3-4778-80ea-4e5a844220f1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:50.296555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cfab0-87f3-4778-80ea-4e5a844220f1 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:50.296555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1391/2773] 10.222.0.83 () {54 vars in 921 bytes} [Wed Aug 24 11:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:40:50.299974 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d0d94-27a7-49d8-b3e8-62c39d2d5db1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:50.301184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d0d94-27a7-49d8-b3e8-62c39d2d5db1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:50.301471 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d0d94-27a7-49d8-b3e8-62c39d2d5db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:50.324261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-123d0d94-27a7-49d8-b3e8-62c39d2d5db1 None None] MFA Rules not processed for user `32a28f9fb3134f33819e53e0c72c63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:40:50.325476 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-123d0d94-27a7-49d8-b3e8-62c39d2d5db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:50.335021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1383/2774] 10.222.0.83 () {58 vars in 1018 bytes} [Wed Aug 24 11:40:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:40:50.345579 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cfab0-87f3-4778-80ea-4e5a844220f1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:50.357268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cfab0-87f3-4778-80ea-4e5a844220f1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:50.360844 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d874f5-75b7-4fa0-bb93-67d7f584f157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:50.361540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d874f5-75b7-4fa0-bb93-67d7f584f157 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:50.361815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d874f5-75b7-4fa0-bb93-67d7f584f157 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:50.362210 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d874f5-75b7-4fa0-bb93-67d7f584f157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:50.373536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d874f5-75b7-4fa0-bb93-67d7f584f157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:50.375012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d874f5-75b7-4fa0-bb93-67d7f584f157 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:50.386119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1392/2775] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:40:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:40:52.916375 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472255dd-8f80-471f-932b-9f636c4459dd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:52.916890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472255dd-8f80-471f-932b-9f636c4459dd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:52.916890 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472255dd-8f80-471f-932b-9f636c4459dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:52.936133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-472255dd-8f80-471f-932b-9f636c4459dd None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:40:52.937340 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-472255dd-8f80-471f-932b-9f636c4459dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:52.994657 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1384/2776] 10.222.0.83 () {58 vars in 1009 bytes} [Wed Aug 24 11:40:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:40:53.007760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d874f5-75b7-4fa0-bb93-67d7f584f157 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:53.009279 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d2d874f5-75b7-4fa0-bb93-67d7f584f157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:53.093931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d874f5-75b7-4fa0-bb93-67d7f584f157 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:53.099339 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a86709-e55c-4c6e-a894-3450f0f21205 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:53.099948 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a86709-e55c-4c6e-a894-3450f0f21205 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:53.100226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a86709-e55c-4c6e-a894-3450f0f21205 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:53.100494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a86709-e55c-4c6e-a894-3450f0f21205 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:53.106740 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:53.109154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a86709-e55c-4c6e-a894-3450f0f21205 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:53.110839 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a86709-e55c-4c6e-a894-3450f0f21205 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:53.128991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1393/2777] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:40:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:41:02.987312 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=65 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:03.168087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:06.724532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472255dd-8f80-471f-932b-9f636c4459dd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:06.733947 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472255dd-8f80-471f-932b-9f636c4459dd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:06.737441 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57923af6-f4c6-46fd-aedb-df197db0d881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:06.738072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57923af6-f4c6-46fd-aedb-df197db0d881 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:06.738352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57923af6-f4c6-46fd-aedb-df197db0d881 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:06.738623 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57923af6-f4c6-46fd-aedb-df197db0d881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:06.740334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-57923af6-f4c6-46fd-aedb-df197db0d881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:06.756651 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57923af6-f4c6-46fd-aedb-df197db0d881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:06.758149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57923af6-f4c6-46fd-aedb-df197db0d881 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:06.766796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1385/2778] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:41:13.359221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8e0ec-8c10-4617-bbd6-fb18a86aee73 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:13.359707 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8e0ec-8c10-4617-bbd6-fb18a86aee73 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:13.359817 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8e0ec-8c10-4617-bbd6-fb18a86aee73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:13.380604 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-94f8e0ec-8c10-4617-bbd6-fb18a86aee73 None None] MFA Rules not processed for user `f9d0b6662bf947cba853388819a933b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:13.381977 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-94f8e0ec-8c10-4617-bbd6-fb18a86aee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:13.389558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1394/2779] 10.222.0.83 () {58 vars in 967 bytes} [Wed Aug 24 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:13.402662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57923af6-f4c6-46fd-aedb-df197db0d881 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:13.415522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57923af6-f4c6-46fd-aedb-df197db0d881 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:13.420464 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921e196-a0ab-4541-bf27-9c7960f5ff7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12eafcbf5c98425782138d5363db9a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:13.421446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921e196-a0ab-4541-bf27-9c7960f5ff7c service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:13.421446 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921e196-a0ab-4541-bf27-9c7960f5ff7c service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:13.421654 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921e196-a0ab-4541-bf27-9c7960f5ff7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:13.431095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921e196-a0ab-4541-bf27-9c7960f5ff7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:13.432679 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921e196-a0ab-4541-bf27-9c7960f5ff7c service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:13.444650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1386/2780] 10.222.0.83 () {60 vars in 1384 bytes} [Wed Aug 24 11:41:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:41:23.374739 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:23.437808 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=12 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:20.229659 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8e0ec-8c10-4617-bbd6-fb18a86aee73 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:20.231162 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-94f8e0ec-8c10-4617-bbd6-fb18a86aee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:20.242931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8e0ec-8c10-4617-bbd6-fb18a86aee73 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:20.246532 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38013ef-6b5b-4762-af73-5a5f96b6e781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:20.247147 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38013ef-6b5b-4762-af73-5a5f96b6e781 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:20.247493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38013ef-6b5b-4762-af73-5a5f96b6e781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:20.247765 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38013ef-6b5b-4762-af73-5a5f96b6e781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:20.249493 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d38013ef-6b5b-4762-af73-5a5f96b6e781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:20.316538 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38013ef-6b5b-4762-af73-5a5f96b6e781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:20.318032 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38013ef-6b5b-4762-af73-5a5f96b6e781 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:20.328719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1395/2781] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:42:23.934241 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921e196-a0ab-4541-bf27-9c7960f5ff7c service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:23.935939 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b921e196-a0ab-4541-bf27-9c7960f5ff7c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:23.950015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921e196-a0ab-4541-bf27-9c7960f5ff7c service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:23.955127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:23.955640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:23.955915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:23.956184 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] PATH_INFO: `/v3/users/4dfbe3078d714129b5a6e6e45dbc0b25` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:23.957442 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dfbe3078d714129b5a6e6e45dbc0b25)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:23.959512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:24.164606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dfbe3078d714129b5a6e6e45dbc0b25'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.165253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dfbe3078d714129b5a6e6e45dbc0b25'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.165857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dfbe3078d714129b5a6e6e45dbc0b25'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.246250 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dfbe3078d714129b5a6e6e45dbc0b25'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.254796 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9da8af98-9620-48f6-8ae1-9f415a491733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dfbe3078d714129b5a6e6e45dbc0b25'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.272975 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1387/2782] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:42:23 2022] DELETE /identity/v3/users/4dfbe3078d714129b5a6e6e45dbc0b25 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:24.542043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38013ef-6b5b-4762-af73-5a5f96b6e781 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:24.554193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38013ef-6b5b-4762-af73-5a5f96b6e781 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:24.613727 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:24.614394 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:24.614677 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:24.614950 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] PATH_INFO: `/v3/projects/de6ccf52d6394c01859a5ccf0774f2d8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:24.616087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de6ccf52d6394c01859a5ccf0774f2d8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:24.618080 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:24.757965 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de6ccf52d6394c01859a5ccf0774f2d8', 'request_id': 'req-e8c991e9-0e8d-4fcb-b729-80562aad966d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.763397 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de6ccf52d6394c01859a5ccf0774f2d8', 'request_id': 'req-e8c991e9-0e8d-4fcb-b729-80562aad966d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.869271 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de6ccf52d6394c01859a5ccf0774f2d8', 'request_id': 'req-e8c991e9-0e8d-4fcb-b729-80562aad966d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.881555 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] The token cache is being invalidate because project de6ccf52d6394c01859a5ccf0774f2d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:42:24.881843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.882508 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8c991e9-0e8d-4fcb-b729-80562aad966d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:24.884120 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1396/2783] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:42:24 2022] DELETE /identity/v3/projects/de6ccf52d6394c01859a5ccf0774f2d8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:24.888989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf26b0e-f424-4532-b9ac-3d0c088242b1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:24.889305 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf26b0e-f424-4532-b9ac-3d0c088242b1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:24.889584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf26b0e-f424-4532-b9ac-3d0c088242b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:24.910331 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cbf26b0e-f424-4532-b9ac-3d0c088242b1 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:24.911554 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cbf26b0e-f424-4532-b9ac-3d0c088242b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:24.969576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1388/2784] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:24.977350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff0d7b-d5b6-4459-938f-207b83a9e9ed None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:24.977640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff0d7b-d5b6-4459-938f-207b83a9e9ed None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:24.977978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff0d7b-d5b6-4459-938f-207b83a9e9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:24.998326 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-07ff0d7b-d5b6-4459-938f-207b83a9e9ed None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:24.999552 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-07ff0d7b-d5b6-4459-938f-207b83a9e9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:25.008043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1397/2785] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:25.012265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf26b0e-f424-4532-b9ac-3d0c088242b1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:25.021960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf26b0e-f424-4532-b9ac-3d0c088242b1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:25.026450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee77c5a-6a5d-45d8-b2ff-ebf8829257a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:25.027037 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee77c5a-6a5d-45d8-b2ff-ebf8829257a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:25.027313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee77c5a-6a5d-45d8-b2ff-ebf8829257a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:25.027580 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee77c5a-6a5d-45d8-b2ff-ebf8829257a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:25.028214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee77c5a-6a5d-45d8-b2ff-ebf8829257a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:42:25.028509 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee77c5a-6a5d-45d8-b2ff-ebf8829257a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:25.030124 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee77c5a-6a5d-45d8-b2ff-ebf8829257a8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:25.042216 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1389/2786] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:42:25.046359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff0d7b-d5b6-4459-938f-207b83a9e9ed None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:25.060741 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff0d7b-d5b6-4459-938f-207b83a9e9ed None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:25.068139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d929e-f899-49d7-9cef-53d3270467d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:25.068900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d929e-f899-49d7-9cef-53d3270467d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:25.069380 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d929e-f899-49d7-9cef-53d3270467d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:25.069726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d929e-f899-49d7-9cef-53d3270467d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:25.070410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d929e-f899-49d7-9cef-53d3270467d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:25.072342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d929e-f899-49d7-9cef-53d3270467d2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:25.089289 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:25.226691 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1398/2787] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:42:25 2022] POST /identity/v3/projects => generated 363 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:42:25.230965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee77c5a-6a5d-45d8-b2ff-ebf8829257a8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:25.241293 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee77c5a-6a5d-45d8-b2ff-ebf8829257a8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:25.298325 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69518edb-5812-4923-9150-05d9ac7462a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:25.298923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69518edb-5812-4923-9150-05d9ac7462a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:25.299201 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69518edb-5812-4923-9150-05d9ac7462a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:25.299469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69518edb-5812-4923-9150-05d9ac7462a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:25.300104 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69518edb-5812-4923-9150-05d9ac7462a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:25.302062 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69518edb-5812-4923-9150-05d9ac7462a4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:25.309842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:25.389332 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1390/2788] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:42:25 2022] POST /identity/v3/users => generated 282 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:42:25.394154 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d929e-f899-49d7-9cef-53d3270467d2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:25.404733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d929e-f899-49d7-9cef-53d3270467d2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:25.410770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df6618-2b02-4cd0-9ac6-823396105abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:25.410770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df6618-2b02-4cd0-9ac6-823396105abe admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:25.411107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df6618-2b02-4cd0-9ac6-823396105abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:25.411268 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df6618-2b02-4cd0-9ac6-823396105abe admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:25.411848 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df6618-2b02-4cd0-9ac6-823396105abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:25.413463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df6618-2b02-4cd0-9ac6-823396105abe admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:25.424136 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1399/2789] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:42:25.430618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69518edb-5812-4923-9150-05d9ac7462a4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:25.441081 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69518edb-5812-4923-9150-05d9ac7462a4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:25.445878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c733d-ff9f-45e8-b93d-0025b35d24db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:25.446606 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c733d-ff9f-45e8-b93d-0025b35d24db admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:25.446988 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c733d-ff9f-45e8-b93d-0025b35d24db admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:25.447370 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c733d-ff9f-45e8-b93d-0025b35d24db admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:25.448137 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c733d-ff9f-45e8-b93d-0025b35d24db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:25.449864 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c733d-ff9f-45e8-b93d-0025b35d24db admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:25.462017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1391/2790] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:42:25.466460 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df6618-2b02-4cd0-9ac6-823396105abe admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:25.477450 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df6618-2b02-4cd0-9ac6-823396105abe admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:25.482142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39681c6-02da-4d08-9f6c-9c3e394ddf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:25.482820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39681c6-02da-4d08-9f6c-9c3e394ddf06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:25.483099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39681c6-02da-4d08-9f6c-9c3e394ddf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:25.483374 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39681c6-02da-4d08-9f6c-9c3e394ddf06 admin admin] PATH_INFO: `/v3/projects/e16c5e57c9364f44a7f71679a7e6a2ca/users/be85b97d936d436cb774a5d505095f7e/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:25.498157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39681c6-02da-4d08-9f6c-9c3e394ddf06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e16c5e57c9364f44a7f71679a7e6a2ca, user_id=be85b97d936d436cb774a5d505095f7e, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:25.501849 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39681c6-02da-4d08-9f6c-9c3e394ddf06 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:25.586711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1400/2791] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:42:25 2022] PUT /identity/v3/projects/e16c5e57c9364f44a7f71679a7e6a2ca/users/be85b97d936d436cb774a5d505095f7e/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:25.601412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920d99-8e94-40c4-b00e-7826ce655daf None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:25.601955 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920d99-8e94-40c4-b00e-7826ce655daf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:25.602247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920d99-8e94-40c4-b00e-7826ce655daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:25.630513 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-14920d99-8e94-40c4-b00e-7826ce655daf None None] MFA Rules not processed for user `be85b97d936d436cb774a5d505095f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:25.631782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-14920d99-8e94-40c4-b00e-7826ce655daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:25.698011 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1392/2792] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:25.702825 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39681c6-02da-4d08-9f6c-9c3e394ddf06 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:25.765461 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39681c6-02da-4d08-9f6c-9c3e394ddf06 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:25.770694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d638e1c-d73c-4278-a30a-fa5b018d6022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:25.771293 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d638e1c-d73c-4278-a30a-fa5b018d6022 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:25.771571 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d638e1c-d73c-4278-a30a-fa5b018d6022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:25.771866 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d638e1c-d73c-4278-a30a-fa5b018d6022 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:25.772500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d638e1c-d73c-4278-a30a-fa5b018d6022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:25.774448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d638e1c-d73c-4278-a30a-fa5b018d6022 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:25.933466 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1401/2793] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:42:25 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:42:25.937584 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920d99-8e94-40c4-b00e-7826ce655daf None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:25.948917 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920d99-8e94-40c4-b00e-7826ce655daf None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:26.004181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648e3f2-2695-49ef-aaea-1e013c1bb062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:26.004869 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648e3f2-2695-49ef-aaea-1e013c1bb062 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:26.005032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648e3f2-2695-49ef-aaea-1e013c1bb062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:26.005297 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648e3f2-2695-49ef-aaea-1e013c1bb062 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:26.005954 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648e3f2-2695-49ef-aaea-1e013c1bb062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:26.007856 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648e3f2-2695-49ef-aaea-1e013c1bb062 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:26.169434 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1393/2794] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:42:25 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:42:26.173751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d638e1c-d73c-4278-a30a-fa5b018d6022 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:26.184274 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d638e1c-d73c-4278-a30a-fa5b018d6022 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:26.189589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f7be2-c949-4b09-830d-af141931389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:26.190216 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f7be2-c949-4b09-830d-af141931389d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:26.190500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f7be2-c949-4b09-830d-af141931389d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:26.190778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f7be2-c949-4b09-830d-af141931389d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:26.191429 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f7be2-c949-4b09-830d-af141931389d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:26.193043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f7be2-c949-4b09-830d-af141931389d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:26.203641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1402/2795] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:42:26.207930 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648e3f2-2695-49ef-aaea-1e013c1bb062 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:26.218344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648e3f2-2695-49ef-aaea-1e013c1bb062 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:26.223613 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d80dfe-63a0-4853-a334-f525c961c835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:26.224363 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d80dfe-63a0-4853-a334-f525c961c835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:26.224643 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d80dfe-63a0-4853-a334-f525c961c835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:26.224915 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d80dfe-63a0-4853-a334-f525c961c835 admin admin] PATH_INFO: `/v3/projects/f5f01263bc8344968a247507c7c6de4a/users/dab4b27f5ae64d54932380c6052303d9/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:26.240610 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d80dfe-63a0-4853-a334-f525c961c835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f01263bc8344968a247507c7c6de4a, user_id=dab4b27f5ae64d54932380c6052303d9, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:26.243283 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d80dfe-63a0-4853-a334-f525c961c835 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:26.352202 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1394/2796] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:42:26 2022] PUT /identity/v3/projects/f5f01263bc8344968a247507c7c6de4a/users/dab4b27f5ae64d54932380c6052303d9/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:26.357108 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f7be2-c949-4b09-830d-af141931389d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:26.368306 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f7be2-c949-4b09-830d-af141931389d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:26.425070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb4773-4ce7-4740-ade7-2d87175d7135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:26.425612 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb4773-4ce7-4740-ade7-2d87175d7135 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:26.425913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb4773-4ce7-4740-ade7-2d87175d7135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:26.426187 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb4773-4ce7-4740-ade7-2d87175d7135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:26.426843 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb4773-4ce7-4740-ade7-2d87175d7135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:26.428513 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb4773-4ce7-4740-ade7-2d87175d7135 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:26.439401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1403/2797] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:42:26.443843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d80dfe-63a0-4853-a334-f525c961c835 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:26.454295 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d80dfe-63a0-4853-a334-f525c961c835 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:26.465951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4c491-85e1-4e8e-a040-223056a6843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:26.465951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4c491-85e1-4e8e-a040-223056a6843b admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:26.465951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4c491-85e1-4e8e-a040-223056a6843b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:26.465951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4c491-85e1-4e8e-a040-223056a6843b admin admin] PATH_INFO: `/v3/projects/f5f01263bc8344968a247507c7c6de4a/users/dab4b27f5ae64d54932380c6052303d9/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:26.466383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4c491-85e1-4e8e-a040-223056a6843b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f01263bc8344968a247507c7c6de4a, user_id=dab4b27f5ae64d54932380c6052303d9, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:26.466383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4c491-85e1-4e8e-a040-223056a6843b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:26.476291 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1395/2798] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:42:26 2022] PUT /identity/v3/projects/f5f01263bc8344968a247507c7c6de4a/users/dab4b27f5ae64d54932380c6052303d9/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:26.538690 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024245b-37e5-4b20-879c-efa8de9c58a5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:26.538976 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024245b-37e5-4b20-879c-efa8de9c58a5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:26.539312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024245b-37e5-4b20-879c-efa8de9c58a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:26.567878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4024245b-37e5-4b20-879c-efa8de9c58a5 None None] MFA Rules not processed for user `dab4b27f5ae64d54932380c6052303d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:26.569105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4024245b-37e5-4b20-879c-efa8de9c58a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:26.588746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:26.638619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1404/2799] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:42:26.650770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4c491-85e1-4e8e-a040-223056a6843b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:26.652516 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-16f4c491-85e1-4e8e-a040-223056a6843b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:26.675662 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:26.710468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4c491-85e1-4e8e-a040-223056a6843b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:26.715356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d04c0-ef81-4105-a44f-9818fb7cc57c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:26.715356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d04c0-ef81-4105-a44f-9818fb7cc57c service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:26.715356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d04c0-ef81-4105-a44f-9818fb7cc57c service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:26.715356 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d04c0-ef81-4105-a44f-9818fb7cc57c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:26.725031 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d04c0-ef81-4105-a44f-9818fb7cc57c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:26.726511 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d04c0-ef81-4105-a44f-9818fb7cc57c service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:26.736454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1396/2800] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:27.021494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024245b-37e5-4b20-879c-efa8de9c58a5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:27.031427 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024245b-37e5-4b20-879c-efa8de9c58a5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:27.035035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d59e46-bf9d-4d7c-86b1-f5a859e4ecc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:27.035715 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d59e46-bf9d-4d7c-86b1-f5a859e4ecc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:27.035998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d59e46-bf9d-4d7c-86b1-f5a859e4ecc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:27.036267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d59e46-bf9d-4d7c-86b1-f5a859e4ecc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:27.106226 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d59e46-bf9d-4d7c-86b1-f5a859e4ecc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:27.107672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d59e46-bf9d-4d7c-86b1-f5a859e4ecc4 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:27.116698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1405/2801] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:30.724501 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d04c0-ef81-4105-a44f-9818fb7cc57c service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:30.788504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d04c0-ef81-4105-a44f-9818fb7cc57c service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:30.793083 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:30.793833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:30.794153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:30.794478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] PATH_INFO: `/v3/users/be85b97d936d436cb774a5d505095f7e` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:30.795782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be85b97d936d436cb774a5d505095f7e)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:30.797704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:31.037409 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be85b97d936d436cb774a5d505095f7e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:31.038101 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be85b97d936d436cb774a5d505095f7e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:31.038718 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be85b97d936d436cb774a5d505095f7e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:31.132248 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be85b97d936d436cb774a5d505095f7e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:31.140108 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be85b97d936d436cb774a5d505095f7e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:31.158745 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1397/2802] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:42:30 2022] DELETE /identity/v3/users/be85b97d936d436cb774a5d505095f7e => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:31.163743 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b9a3-5cbd-4397-a049-056b49ebe42c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:31.164266 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b9a3-5cbd-4397-a049-056b49ebe42c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:31.164377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b9a3-5cbd-4397-a049-056b49ebe42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:31.183514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a831b9a3-5cbd-4397-a049-056b49ebe42c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:31.184711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a831b9a3-5cbd-4397-a049-056b49ebe42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:31.243158 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1406/2803] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:31.253284 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:31.254732 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:31.300676 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:31.312221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19b125-a8b5-44d7-b459-1d0694d3c090 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:31.319635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacdc94-5b2c-428b-ab9c-3ab630b6ed7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:31.319635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacdc94-5b2c-428b-ab9c-3ab630b6ed7a service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:31.319635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacdc94-5b2c-428b-ab9c-3ab630b6ed7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:31.319635 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacdc94-5b2c-428b-ab9c-3ab630b6ed7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:31.326227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacdc94-5b2c-428b-ab9c-3ab630b6ed7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:31.327669 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacdc94-5b2c-428b-ab9c-3ab630b6ed7a service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:31.337951 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1398/2804] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:31.779415 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831b9a3-5cbd-4397-a049-056b49ebe42c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:31.780907 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a831b9a3-5cbd-4397-a049-056b49ebe42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:31.795209 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831b9a3-5cbd-4397-a049-056b49ebe42c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:31.800105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:31.800759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:31.801055 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:31.801313 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] PATH_INFO: `/v3/projects/e16c5e57c9364f44a7f71679a7e6a2ca` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:31.802498 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e16c5e57c9364f44a7f71679a7e6a2ca)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:31.804480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:31.922614 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e16c5e57c9364f44a7f71679a7e6a2ca', 'request_id': 'req-faa23e02-588a-4569-8449-6b756ba771ae'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:31.924359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:31.929394 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e16c5e57c9364f44a7f71679a7e6a2ca', 'request_id': 'req-faa23e02-588a-4569-8449-6b756ba771ae'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:32.019365 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e16c5e57c9364f44a7f71679a7e6a2ca', 'request_id': 'req-faa23e02-588a-4569-8449-6b756ba771ae'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:32.032155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] The token cache is being invalidate because project e16c5e57c9364f44a7f71679a7e6a2ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:42:32.032448 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:32.033110 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:32.034710 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1407/2805] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:42:31 2022] DELETE /identity/v3/projects/e16c5e57c9364f44a7f71679a7e6a2ca => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:32.039213 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacdc94-5b2c-428b-ab9c-3ab630b6ed7a service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:32.050008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacdc94-5b2c-428b-ab9c-3ab630b6ed7a service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:32.104783 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:32.105320 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:32.105596 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:32.105882 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] PATH_INFO: `/v3/users/dab4b27f5ae64d54932380c6052303d9` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:32.107010 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dab4b27f5ae64d54932380c6052303d9)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:32.108913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:32.278704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dab4b27f5ae64d54932380c6052303d9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:32.279313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dab4b27f5ae64d54932380c6052303d9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:32.279916 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dab4b27f5ae64d54932380c6052303d9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:32.354735 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dab4b27f5ae64d54932380c6052303d9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:32.362855 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1b487ff1-019c-4c57-85f7-f86b6c187362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dab4b27f5ae64d54932380c6052303d9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:32.382161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1399/2806] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:42:32 2022] DELETE /identity/v3/users/dab4b27f5ae64d54932380c6052303d9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:32.843978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:32.845540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:32.860264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa23e02-588a-4569-8449-6b756ba771ae admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:32.919363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:32.919919 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:32.920214 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:32.920471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] PATH_INFO: `/v3/projects/f5f01263bc8344968a247507c7c6de4a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:32.921593 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f01263bc8344968a247507c7c6de4a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:32.924061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:33.117495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f01263bc8344968a247507c7c6de4a', 'request_id': 'req-9098c22e-8284-4be9-9b7f-74e0159ab344'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:33.123238 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f01263bc8344968a247507c7c6de4a', 'request_id': 'req-9098c22e-8284-4be9-9b7f-74e0159ab344'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:33.225694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f01263bc8344968a247507c7c6de4a', 'request_id': 'req-9098c22e-8284-4be9-9b7f-74e0159ab344'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:33.238430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] The token cache is being invalidate because project f5f01263bc8344968a247507c7c6de4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:42:33.238720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:33.239348 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9098c22e-8284-4be9-9b7f-74e0159ab344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:33.240956 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1408/2807] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:42:32 2022] DELETE /identity/v3/projects/f5f01263bc8344968a247507c7c6de4a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:33.245945 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad08a81-7f20-43de-893d-11dd001d3ccb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:33.246467 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad08a81-7f20-43de-893d-11dd001d3ccb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:33.246638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad08a81-7f20-43de-893d-11dd001d3ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:33.267099 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-aad08a81-7f20-43de-893d-11dd001d3ccb None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:33.268313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-aad08a81-7f20-43de-893d-11dd001d3ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:33.327754 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1400/2808] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:33.335963 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307e89f-8f50-49f7-be0a-001f373f3d3f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:33.336244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307e89f-8f50-49f7-be0a-001f373f3d3f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:33.336517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307e89f-8f50-49f7-be0a-001f373f3d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:33.356417 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7307e89f-8f50-49f7-be0a-001f373f3d3f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:33.357632 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7307e89f-8f50-49f7-be0a-001f373f3d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:33.365572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1409/2809] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:33.369729 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad08a81-7f20-43de-893d-11dd001d3ccb None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:33.376391 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:33.380254 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad08a81-7f20-43de-893d-11dd001d3ccb None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:33.385296 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7965895-9601-405b-bc3d-f2320e4c634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:33.385922 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7965895-9601-405b-bc3d-f2320e4c634e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:33.386329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7965895-9601-405b-bc3d-f2320e4c634e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:33.386611 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7965895-9601-405b-bc3d-f2320e4c634e admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:33.387313 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7965895-9601-405b-bc3d-f2320e4c634e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:42:33.387587 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7965895-9601-405b-bc3d-f2320e4c634e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:33.389227 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7965895-9601-405b-bc3d-f2320e4c634e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:33.401983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1401/2810] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:42:33.406645 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307e89f-8f50-49f7-be0a-001f373f3d3f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:33.417250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307e89f-8f50-49f7-be0a-001f373f3d3f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:33.422125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72efbd-6a8a-48cf-956c-6361602f4c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:33.425778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72efbd-6a8a-48cf-956c-6361602f4c12 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:33.425778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72efbd-6a8a-48cf-956c-6361602f4c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:33.425778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72efbd-6a8a-48cf-956c-6361602f4c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:33.425778 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72efbd-6a8a-48cf-956c-6361602f4c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:33.426219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72efbd-6a8a-48cf-956c-6361602f4c12 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:33.517623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1410/2811] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:42:33 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:42:33.521755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7965895-9601-405b-bc3d-f2320e4c634e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:33.532136 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7965895-9601-405b-bc3d-f2320e4c634e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:33.590507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dbc30-9227-4e81-b727-a9edf8c4e8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:33.591116 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dbc30-9227-4e81-b727-a9edf8c4e8ba admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:33.591439 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dbc30-9227-4e81-b727-a9edf8c4e8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:33.591673 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dbc30-9227-4e81-b727-a9edf8c4e8ba admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:33.592301 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dbc30-9227-4e81-b727-a9edf8c4e8ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:33.594260 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dbc30-9227-4e81-b727-a9edf8c4e8ba admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:33.788851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1402/2812] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:42:33 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:42:33.789221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72efbd-6a8a-48cf-956c-6361602f4c12 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:33.789221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72efbd-6a8a-48cf-956c-6361602f4c12 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:33.793161 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5e82a-7de1-4d9d-af08-70af3b4f8930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:33.793687 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5e82a-7de1-4d9d-af08-70af3b4f8930 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:33.793998 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5e82a-7de1-4d9d-af08-70af3b4f8930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:33.794292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5e82a-7de1-4d9d-af08-70af3b4f8930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:33.794945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5e82a-7de1-4d9d-af08-70af3b4f8930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:33.796548 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5e82a-7de1-4d9d-af08-70af3b4f8930 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:33.807247 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1411/2813] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:42:33.811385 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dbc30-9227-4e81-b727-a9edf8c4e8ba admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:33.822047 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dbc30-9227-4e81-b727-a9edf8c4e8ba admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:33.828857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665f552-dac2-4cb6-b96b-2d3ccc8207d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:33.828857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665f552-dac2-4cb6-b96b-2d3ccc8207d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:33.828857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665f552-dac2-4cb6-b96b-2d3ccc8207d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:33.828857 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665f552-dac2-4cb6-b96b-2d3ccc8207d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:33.829311 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665f552-dac2-4cb6-b96b-2d3ccc8207d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:33.830553 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665f552-dac2-4cb6-b96b-2d3ccc8207d1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:33.841082 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1403/2814] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:42:33.845284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5e82a-7de1-4d9d-af08-70af3b4f8930 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:33.855273 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5e82a-7de1-4d9d-af08-70af3b4f8930 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:33.859815 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d839a08-7bef-40fa-937a-4192a627d6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:33.860501 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839a08-7bef-40fa-937a-4192a627d6e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:33.860751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839a08-7bef-40fa-937a-4192a627d6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:33.861026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839a08-7bef-40fa-937a-4192a627d6e8 admin admin] PATH_INFO: `/v3/projects/20fb92a88a904cc9addf45969b7564a7/users/d28c65759a544589b5461f1ccab69d4f/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:33.875758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d839a08-7bef-40fa-937a-4192a627d6e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20fb92a88a904cc9addf45969b7564a7, user_id=d28c65759a544589b5461f1ccab69d4f, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:33.879054 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d839a08-7bef-40fa-937a-4192a627d6e8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:34.008939 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1412/2815] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:42:33 2022] PUT /identity/v3/projects/20fb92a88a904cc9addf45969b7564a7/users/d28c65759a544589b5461f1ccab69d4f/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:34.014335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b27d0-8ef9-4594-8e1d-27f8fd2c6801 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:34.014619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b27d0-8ef9-4594-8e1d-27f8fd2c6801 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:34.014912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b27d0-8ef9-4594-8e1d-27f8fd2c6801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:34.033862 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5b4b27d0-8ef9-4594-8e1d-27f8fd2c6801 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:34.035089 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5b4b27d0-8ef9-4594-8e1d-27f8fd2c6801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:34.097334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1404/2816] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:42:34.107122 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d839a08-7bef-40fa-937a-4192a627d6e8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:34.108882 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4d839a08-7bef-40fa-937a-4192a627d6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:34.116068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:34.166232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d839a08-7bef-40fa-937a-4192a627d6e8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:34.170185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e87cb-5e32-49a4-878f-336ddce4ec55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:34.170521 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e87cb-5e32-49a4-878f-336ddce4ec55 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:34.170797 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e87cb-5e32-49a4-878f-336ddce4ec55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:34.171068 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e87cb-5e32-49a4-878f-336ddce4ec55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:34.181143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e87cb-5e32-49a4-878f-336ddce4ec55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:34.182644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e87cb-5e32-49a4-878f-336ddce4ec55 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:34.193572 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1413/2817] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:35.633901 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997d7cf-cf4b-441b-b1ed-5744603c3255 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:35.634453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997d7cf-cf4b-441b-b1ed-5744603c3255 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:35.634453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997d7cf-cf4b-441b-b1ed-5744603c3255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:35.663575 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a997d7cf-cf4b-441b-b1ed-5744603c3255 None None] MFA Rules not processed for user `d28c65759a544589b5461f1ccab69d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:35.664853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a997d7cf-cf4b-441b-b1ed-5744603c3255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:35.728487 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:35.732640 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1405/2818] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:35.737597 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e87cb-5e32-49a4-878f-336ddce4ec55 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:35.747661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e87cb-5e32-49a4-878f-336ddce4ec55 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:35.753157 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc0bb4-9a51-4157-b3d3-b624fb0cf54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:35.753762 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc0bb4-9a51-4157-b3d3-b624fb0cf54d admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:35.754074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc0bb4-9a51-4157-b3d3-b624fb0cf54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:35.754349 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc0bb4-9a51-4157-b3d3-b624fb0cf54d admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:35.754991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc0bb4-9a51-4157-b3d3-b624fb0cf54d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:35.756900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc0bb4-9a51-4157-b3d3-b624fb0cf54d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:35.844520 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1414/2819] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:42:35 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:42:35.848666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997d7cf-cf4b-441b-b1ed-5744603c3255 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:35.859336 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997d7cf-cf4b-441b-b1ed-5744603c3255 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:35.915304 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6211-484a-4510-95de-f13c84f4a035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:35.915906 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6211-484a-4510-95de-f13c84f4a035 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:35.916185 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6211-484a-4510-95de-f13c84f4a035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:35.916457 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6211-484a-4510-95de-f13c84f4a035 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:35.917088 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a6211-484a-4510-95de-f13c84f4a035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:35.919036 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a6211-484a-4510-95de-f13c84f4a035 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:36.025360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1406/2820] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:42:35 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:42:36.029431 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc0bb4-9a51-4157-b3d3-b624fb0cf54d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:36.040439 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc0bb4-9a51-4157-b3d3-b624fb0cf54d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:36.045076 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e797ff2-0688-4c17-8e7c-b5a2b62c8de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:36.045672 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e797ff2-0688-4c17-8e7c-b5a2b62c8de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:36.045997 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e797ff2-0688-4c17-8e7c-b5a2b62c8de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:36.046275 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e797ff2-0688-4c17-8e7c-b5a2b62c8de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:36.046929 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e797ff2-0688-4c17-8e7c-b5a2b62c8de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:36.049195 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e797ff2-0688-4c17-8e7c-b5a2b62c8de5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:36.059786 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1415/2821] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:42:36.063911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6211-484a-4510-95de-f13c84f4a035 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:36.074143 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6211-484a-4510-95de-f13c84f4a035 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:36.078923 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0aaee8-9a60-478d-8a55-ff696d7f09b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:36.079560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0aaee8-9a60-478d-8a55-ff696d7f09b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:36.079837 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0aaee8-9a60-478d-8a55-ff696d7f09b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:36.080109 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0aaee8-9a60-478d-8a55-ff696d7f09b6 admin admin] PATH_INFO: `/v3/projects/f800c5d7718747d19d209b30b1b75121/users/4837fdeadaf749388c192512b56beb2b/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:36.094770 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0aaee8-9a60-478d-8a55-ff696d7f09b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f800c5d7718747d19d209b30b1b75121, user_id=4837fdeadaf749388c192512b56beb2b, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:36.097934 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0aaee8-9a60-478d-8a55-ff696d7f09b6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:36.218167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1407/2822] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:42:36 2022] PUT /identity/v3/projects/f800c5d7718747d19d209b30b1b75121/users/4837fdeadaf749388c192512b56beb2b/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:36.222935 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e797ff2-0688-4c17-8e7c-b5a2b62c8de5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:36.233634 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e797ff2-0688-4c17-8e7c-b5a2b62c8de5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:36.291036 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87973dd1-ec2d-4053-b82a-2590a8a93d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:36.291653 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87973dd1-ec2d-4053-b82a-2590a8a93d22 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:36.291940 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87973dd1-ec2d-4053-b82a-2590a8a93d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:36.292215 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87973dd1-ec2d-4053-b82a-2590a8a93d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:36.292869 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87973dd1-ec2d-4053-b82a-2590a8a93d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:36.294581 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87973dd1-ec2d-4053-b82a-2590a8a93d22 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:36.305488 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1416/2823] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:42:36.311507 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0aaee8-9a60-478d-8a55-ff696d7f09b6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:36.320179 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0aaee8-9a60-478d-8a55-ff696d7f09b6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:36.325059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655db9f-c94f-4669-b19a-dce16241cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:36.325797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655db9f-c94f-4669-b19a-dce16241cb52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:36.326178 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655db9f-c94f-4669-b19a-dce16241cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:36.326541 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655db9f-c94f-4669-b19a-dce16241cb52 admin admin] PATH_INFO: `/v3/projects/f800c5d7718747d19d209b30b1b75121/users/4837fdeadaf749388c192512b56beb2b/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:36.328454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655db9f-c94f-4669-b19a-dce16241cb52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f800c5d7718747d19d209b30b1b75121, user_id=4837fdeadaf749388c192512b56beb2b, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:36.331663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655db9f-c94f-4669-b19a-dce16241cb52 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:36.342200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1408/2824] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:42:36 2022] PUT /identity/v3/projects/f800c5d7718747d19d209b30b1b75121/users/4837fdeadaf749388c192512b56beb2b/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:37.682361 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5281d1-7c18-4fa4-a4e1-77aebdbf5c2c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:37.682859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5281d1-7c18-4fa4-a4e1-77aebdbf5c2c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:37.682987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5281d1-7c18-4fa4-a4e1-77aebdbf5c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:37.715758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2c5281d1-7c18-4fa4-a4e1-77aebdbf5c2c None None] MFA Rules not processed for user `4837fdeadaf749388c192512b56beb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:37.718224 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2c5281d1-7c18-4fa4-a4e1-77aebdbf5c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:37.787132 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:37.798887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1417/2825] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:42:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:37.810790 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655db9f-c94f-4669-b19a-dce16241cb52 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:37.812483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8655db9f-c94f-4669-b19a-dce16241cb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:37.870327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655db9f-c94f-4669-b19a-dce16241cb52 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:37.873948 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564563d-720d-4c42-8525-b5aaf593afad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:37.874678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564563d-720d-4c42-8525-b5aaf593afad service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:37.874878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564563d-720d-4c42-8525-b5aaf593afad service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:37.875406 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564563d-720d-4c42-8525-b5aaf593afad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:37.933942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564563d-720d-4c42-8525-b5aaf593afad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:37.935381 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564563d-720d-4c42-8525-b5aaf593afad service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:37.944279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1409/2826] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:40.682987 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5281d1-7c18-4fa4-a4e1-77aebdbf5c2c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:40.703051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5281d1-7c18-4fa4-a4e1-77aebdbf5c2c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:40.707771 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96a4e2-c797-490a-84c1-10d3d75f066e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:40.708370 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96a4e2-c797-490a-84c1-10d3d75f066e service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:40.708644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96a4e2-c797-490a-84c1-10d3d75f066e service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:40.708938 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96a4e2-c797-490a-84c1-10d3d75f066e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:40.739323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96a4e2-c797-490a-84c1-10d3d75f066e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:40.740754 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96a4e2-c797-490a-84c1-10d3d75f066e service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:40.750576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1418/2827] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:47.940292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:50.746018 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=22 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:58.328794 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564563d-720d-4c42-8525-b5aaf593afad service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:58.333757 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6564563d-720d-4c42-8525-b5aaf593afad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:58.444198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564563d-720d-4c42-8525-b5aaf593afad service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:58.450590 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe91b9-6d6e-4e36-91ba-833ced1aa4ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:58.451299 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe91b9-6d6e-4e36-91ba-833ced1aa4ff service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:58.451586 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe91b9-6d6e-4e36-91ba-833ced1aa4ff service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:58.451866 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe91b9-6d6e-4e36-91ba-833ced1aa4ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:58.454919 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8efe91b9-6d6e-4e36-91ba-833ced1aa4ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:58.472470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe91b9-6d6e-4e36-91ba-833ced1aa4ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:58.472470 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe91b9-6d6e-4e36-91ba-833ced1aa4ff service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:58.480626 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1410/2828] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:59.144931 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96a4e2-c797-490a-84c1-10d3d75f066e service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:59.214042 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96a4e2-c797-490a-84c1-10d3d75f066e service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:59.219168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:59.219861 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:59.220188 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:59.220531 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] PATH_INFO: `/v3/users/d28c65759a544589b5461f1ccab69d4f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:59.221723 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28c65759a544589b5461f1ccab69d4f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:59.223894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:59.426823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28c65759a544589b5461f1ccab69d4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:59.427518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28c65759a544589b5461f1ccab69d4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:59.428144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28c65759a544589b5461f1ccab69d4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:59.436422 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28c65759a544589b5461f1ccab69d4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:59.448961 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28c65759a544589b5461f1ccab69d4f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:59.536912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1419/2829] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:42:59 2022] DELETE /identity/v3/users/d28c65759a544589b5461f1ccab69d4f => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:59.820530 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe91b9-6d6e-4e36-91ba-833ced1aa4ff service placement] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:59.834001 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe91b9-6d6e-4e36-91ba-833ced1aa4ff service placement] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:59.893601 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:59.894435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:59.894716 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:59.894989 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] PATH_INFO: `/v3/projects/20fb92a88a904cc9addf45969b7564a7` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:59.896110 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=20fb92a88a904cc9addf45969b7564a7)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:59.898046 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:00.015892 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20fb92a88a904cc9addf45969b7564a7', 'request_id': 'req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.017231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:00.022743 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20fb92a88a904cc9addf45969b7564a7', 'request_id': 'req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.029726 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20fb92a88a904cc9addf45969b7564a7', 'request_id': 'req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.161466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] The token cache is being invalidate because project 20fb92a88a904cc9addf45969b7564a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:43:00.161683 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.162330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.163811 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1411/2830] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:42:59 2022] DELETE /identity/v3/projects/20fb92a88a904cc9addf45969b7564a7 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:00.168411 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:00.169860 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:00.184175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29afd7-a332-4558-80f2-52215ceaa67b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:00.246353 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:00.247177 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:00.247456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:00.247731 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] PATH_INFO: `/v3/users/4837fdeadaf749388c192512b56beb2b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:00.248894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4837fdeadaf749388c192512b56beb2b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:00.251170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:00.274058 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:00.483099 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4837fdeadaf749388c192512b56beb2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.483952 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4837fdeadaf749388c192512b56beb2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.484726 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4837fdeadaf749388c192512b56beb2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.493276 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4837fdeadaf749388c192512b56beb2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.506412 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-03e8faaa-1285-4bc6-93cd-bbfa4ef87dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4837fdeadaf749388c192512b56beb2b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:00.608237 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1420/2831] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:43:00 2022] DELETE /identity/v3/users/4837fdeadaf749388c192512b56beb2b => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:00.865694 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:00.870207 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:00.880327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30e9a7-9189-49a3-932f-e21f42e4ccaa admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:00.937257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:00.937911 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:00.938197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:00.938474 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] PATH_INFO: `/v3/projects/f800c5d7718747d19d209b30b1b75121` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:00.939596 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f800c5d7718747d19d209b30b1b75121)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:00.941503 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:01.070846 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f800c5d7718747d19d209b30b1b75121', 'request_id': 'req-8bb18def-7a96-4925-aa54-f9615c715a05'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:01.077087 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f800c5d7718747d19d209b30b1b75121', 'request_id': 'req-8bb18def-7a96-4925-aa54-f9615c715a05'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:01.083469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f800c5d7718747d19d209b30b1b75121', 'request_id': 'req-8bb18def-7a96-4925-aa54-f9615c715a05'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:01.189016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] The token cache is being invalidate because project f800c5d7718747d19d209b30b1b75121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:43:01.189339 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:01.189995 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8bb18def-7a96-4925-aa54-f9615c715a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:01.191479 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1412/2832] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:43:00 2022] DELETE /identity/v3/projects/f800c5d7718747d19d209b30b1b75121 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:01.196430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd774a55-05d4-4a6e-84ac-00f8f1242a5f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:01.196909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd774a55-05d4-4a6e-84ac-00f8f1242a5f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:01.197017 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd774a55-05d4-4a6e-84ac-00f8f1242a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:01.228869 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fd774a55-05d4-4a6e-84ac-00f8f1242a5f None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:01.230121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fd774a55-05d4-4a6e-84ac-00f8f1242a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:01.291994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1421/2833] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:01.300731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01056380-12d3-47a5-ac6c-2a848e51c6a2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:01.301021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01056380-12d3-47a5-ac6c-2a848e51c6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:01.301329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01056380-12d3-47a5-ac6c-2a848e51c6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:01.321782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-01056380-12d3-47a5-ac6c-2a848e51c6a2 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:01.323656 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-01056380-12d3-47a5-ac6c-2a848e51c6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:01.331883 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1413/2834] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:01.336175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd774a55-05d4-4a6e-84ac-00f8f1242a5f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:01.346034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd774a55-05d4-4a6e-84ac-00f8f1242a5f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:01.350662 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd5de-7b94-4307-a3ee-8bf99d25243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:01.351253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd5de-7b94-4307-a3ee-8bf99d25243d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:01.351533 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd5de-7b94-4307-a3ee-8bf99d25243d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:01.351807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd5de-7b94-4307-a3ee-8bf99d25243d admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:01.352455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd5de-7b94-4307-a3ee-8bf99d25243d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:43:01.352758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd5de-7b94-4307-a3ee-8bf99d25243d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:01.355253 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd5de-7b94-4307-a3ee-8bf99d25243d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:01.367250 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1422/2835] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:43:01.372288 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01056380-12d3-47a5-ac6c-2a848e51c6a2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:01.381843 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01056380-12d3-47a5-ac6c-2a848e51c6a2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:01.389500 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24dc99-2460-42a4-ad25-54ccbcac8089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:01.390270 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24dc99-2460-42a4-ad25-54ccbcac8089 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:01.391521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24dc99-2460-42a4-ad25-54ccbcac8089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:01.391807 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24dc99-2460-42a4-ad25-54ccbcac8089 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:01.392468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24dc99-2460-42a4-ad25-54ccbcac8089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:01.394403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24dc99-2460-42a4-ad25-54ccbcac8089 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:01.489193 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1414/2836] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:43:01 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:43:01.493221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd5de-7b94-4307-a3ee-8bf99d25243d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:01.504159 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd5de-7b94-4307-a3ee-8bf99d25243d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:01.560441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a00e55-f006-4305-92fa-2d0ed61db12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:01.561041 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a00e55-f006-4305-92fa-2d0ed61db12e admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:01.561320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a00e55-f006-4305-92fa-2d0ed61db12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:01.561588 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a00e55-f006-4305-92fa-2d0ed61db12e admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:01.562282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a00e55-f006-4305-92fa-2d0ed61db12e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:01.564203 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a00e55-f006-4305-92fa-2d0ed61db12e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:01.642363 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1423/2837] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:43:01 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:43:01.646627 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24dc99-2460-42a4-ad25-54ccbcac8089 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:01.657187 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24dc99-2460-42a4-ad25-54ccbcac8089 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:01.661977 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d6774-7aed-47df-bc08-86caa10c73b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:01.662633 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d6774-7aed-47df-bc08-86caa10c73b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:01.662868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d6774-7aed-47df-bc08-86caa10c73b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:01.663142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d6774-7aed-47df-bc08-86caa10c73b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:01.663853 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d6774-7aed-47df-bc08-86caa10c73b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:01.665429 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d6774-7aed-47df-bc08-86caa10c73b3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:01.676578 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1415/2838] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:01.686784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a00e55-f006-4305-92fa-2d0ed61db12e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:01.691398 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a00e55-f006-4305-92fa-2d0ed61db12e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:01.696043 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f0658-3125-41bf-8e78-8eea229219b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:01.696628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f0658-3125-41bf-8e78-8eea229219b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:01.696901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f0658-3125-41bf-8e78-8eea229219b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:01.697208 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f0658-3125-41bf-8e78-8eea229219b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:01.697896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f0658-3125-41bf-8e78-8eea229219b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:01.699499 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f0658-3125-41bf-8e78-8eea229219b0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:01.700800 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:01.710751 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1424/2839] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:01.739329 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d6774-7aed-47df-bc08-86caa10c73b3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:01.750072 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d6774-7aed-47df-bc08-86caa10c73b3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:01.754596 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43665b-32c7-4971-9a45-20d283573dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:01.755229 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43665b-32c7-4971-9a45-20d283573dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:01.755502 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43665b-32c7-4971-9a45-20d283573dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:01.755774 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43665b-32c7-4971-9a45-20d283573dfe admin admin] PATH_INFO: `/v3/projects/4af33f3a1e044739b47caa19ea27a646/users/843430c6efba48bda5371028652e311f/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:01.770169 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43665b-32c7-4971-9a45-20d283573dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af33f3a1e044739b47caa19ea27a646, user_id=843430c6efba48bda5371028652e311f, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:01.776499 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43665b-32c7-4971-9a45-20d283573dfe admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:01.884013 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1416/2840] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:43:01 2022] PUT /identity/v3/projects/4af33f3a1e044739b47caa19ea27a646/users/843430c6efba48bda5371028652e311f/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:01.894774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e113988-fa22-440a-a5f8-8989d5015bc6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:01.895061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e113988-fa22-440a-a5f8-8989d5015bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:01.895344 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e113988-fa22-440a-a5f8-8989d5015bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:01.923838 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7e113988-fa22-440a-a5f8-8989d5015bc6 None None] MFA Rules not processed for user `843430c6efba48bda5371028652e311f`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:01.925185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7e113988-fa22-440a-a5f8-8989d5015bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:01.993553 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1425/2841] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:02.005625 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43665b-32c7-4971-9a45-20d283573dfe admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:02.007520 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4d43665b-32c7-4971-9a45-20d283573dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:02.014265 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:02.064913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43665b-32c7-4971-9a45-20d283573dfe admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:02.068905 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5f33-034d-44fd-83ac-d8b492532c8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:02.069727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5f33-034d-44fd-83ac-d8b492532c8a service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:02.070249 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5f33-034d-44fd-83ac-d8b492532c8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:02.070909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5f33-034d-44fd-83ac-d8b492532c8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:02.079607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd5f33-034d-44fd-83ac-d8b492532c8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:02.081154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd5f33-034d-44fd-83ac-d8b492532c8a service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:02.090666 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1417/2842] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:02.112865 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e113988-fa22-440a-a5f8-8989d5015bc6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:02.176694 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e113988-fa22-440a-a5f8-8989d5015bc6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:02.181399 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:02.182070 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:02.182355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:02.182628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] PATH_INFO: `/v3/users/843430c6efba48bda5371028652e311f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:02.183896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=843430c6efba48bda5371028652e311f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:02.185823 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:02.369851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '843430c6efba48bda5371028652e311f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:02.370554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '843430c6efba48bda5371028652e311f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:02.371178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '843430c6efba48bda5371028652e311f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:02.378357 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '843430c6efba48bda5371028652e311f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:02.390549 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '843430c6efba48bda5371028652e311f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:02.457945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1426/2843] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:43:02 2022] DELETE /identity/v3/users/843430c6efba48bda5371028652e311f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:02.462485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246440-1fff-482d-bbf7-f6c27950137d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:02.462780 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246440-1fff-482d-bbf7-f6c27950137d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:02.463629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246440-1fff-482d-bbf7-f6c27950137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:02.483966 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a8246440-1fff-482d-bbf7-f6c27950137d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:02.485215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a8246440-1fff-482d-bbf7-f6c27950137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:02.543710 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1418/2844] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:43:02.553688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:02.555143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:02.562379 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:02.613148 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c5ee-c82b-4398-ab05-1a4fa63ed442 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:02.616784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d774c-ce9e-4444-9234-01efd2c1590a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:02.617392 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d774c-ce9e-4444-9234-01efd2c1590a service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:02.617670 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d774c-ce9e-4444-9234-01efd2c1590a service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:02.617973 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d774c-ce9e-4444-9234-01efd2c1590a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:02.628093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d774c-ce9e-4444-9234-01efd2c1590a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:02.629536 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d774c-ce9e-4444-9234-01efd2c1590a service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:02.639400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1427/2845] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:03.082782 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246440-1fff-482d-bbf7-f6c27950137d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:03.084286 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a8246440-1fff-482d-bbf7-f6c27950137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:03.096650 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246440-1fff-482d-bbf7-f6c27950137d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:03.101639 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:03.102526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:03.102918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:03.103080 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] PATH_INFO: `/v3/projects/4af33f3a1e044739b47caa19ea27a646` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:03.104279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af33f3a1e044739b47caa19ea27a646)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:03.106341 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:03.225041 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af33f3a1e044739b47caa19ea27a646', 'request_id': 'req-0983f411-b01c-4592-bb46-ab635034e2e8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:03.231002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af33f3a1e044739b47caa19ea27a646', 'request_id': 'req-0983f411-b01c-4592-bb46-ab635034e2e8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:03.237322 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af33f3a1e044739b47caa19ea27a646', 'request_id': 'req-0983f411-b01c-4592-bb46-ab635034e2e8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:03.342712 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] The token cache is being invalidate because project 4af33f3a1e044739b47caa19ea27a646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:43:03.343008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:03.343643 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0983f411-b01c-4592-bb46-ab635034e2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:03.345200 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1419/2846] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:43:03 2022] DELETE /identity/v3/projects/4af33f3a1e044739b47caa19ea27a646 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:03.350139 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88ce7-bf8c-4f82-b48a-633de10f644a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:03.350430 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88ce7-bf8c-4f82-b48a-633de10f644a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:03.350711 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88ce7-bf8c-4f82-b48a-633de10f644a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:03.371428 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-87c88ce7-bf8c-4f82-b48a-633de10f644a None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:03.372644 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-87c88ce7-bf8c-4f82-b48a-633de10f644a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:03.437393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1428/2847] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:03.445149 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c32e4-9d37-494a-bbe5-5bf2b60bd5dc None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:03.445430 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c32e4-9d37-494a-bbe5-5bf2b60bd5dc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:03.445701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c32e4-9d37-494a-bbe5-5bf2b60bd5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:03.465405 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-268c32e4-9d37-494a-bbe5-5bf2b60bd5dc None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:03.466644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-268c32e4-9d37-494a-bbe5-5bf2b60bd5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:03.474549 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1420/2848] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:03.479594 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88ce7-bf8c-4f82-b48a-633de10f644a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:03.489589 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88ce7-bf8c-4f82-b48a-633de10f644a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:03.494908 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e20c04-da3a-4274-bb38-d5aeb93aeaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:03.495554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e20c04-da3a-4274-bb38-d5aeb93aeaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:03.495832 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e20c04-da3a-4274-bb38-d5aeb93aeaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:03.496104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e20c04-da3a-4274-bb38-d5aeb93aeaf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:03.496760 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e20c04-da3a-4274-bb38-d5aeb93aeaf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:43:03.497062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e20c04-da3a-4274-bb38-d5aeb93aeaf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:03.499123 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e20c04-da3a-4274-bb38-d5aeb93aeaf1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:03.511026 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1429/2849] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:43:03.515352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c32e4-9d37-494a-bbe5-5bf2b60bd5dc None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:03.524717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c32e4-9d37-494a-bbe5-5bf2b60bd5dc None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:03.529854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9c43-ecd8-4aaa-8471-9d4a8dd3f33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:03.530478 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9c43-ecd8-4aaa-8471-9d4a8dd3f33e admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:03.530734 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9c43-ecd8-4aaa-8471-9d4a8dd3f33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:03.530999 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9c43-ecd8-4aaa-8471-9d4a8dd3f33e admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:03.531614 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e9c43-ecd8-4aaa-8471-9d4a8dd3f33e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:03.533486 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e9c43-ecd8-4aaa-8471-9d4a8dd3f33e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:03.626165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1421/2850] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:43:03 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:43:03.630446 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e20c04-da3a-4274-bb38-d5aeb93aeaf1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:03.640576 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e20c04-da3a-4274-bb38-d5aeb93aeaf1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:03.698453 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad8ca6-1da6-4a08-95cc-4dd8b4a90c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:03.699074 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad8ca6-1da6-4a08-95cc-4dd8b4a90c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:03.699383 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad8ca6-1da6-4a08-95cc-4dd8b4a90c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:03.699630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad8ca6-1da6-4a08-95cc-4dd8b4a90c51 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:03.700360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad8ca6-1da6-4a08-95cc-4dd8b4a90c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:03.702337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad8ca6-1da6-4a08-95cc-4dd8b4a90c51 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:03.807523 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1430/2851] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:43:03 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:43:03.812016 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9c43-ecd8-4aaa-8471-9d4a8dd3f33e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:03.817724 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:03.823863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9c43-ecd8-4aaa-8471-9d4a8dd3f33e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:03.828581 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cfb10-5dde-46e0-895b-bc8d1cae66cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:03.829233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cfb10-5dde-46e0-895b-bc8d1cae66cd admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:03.829515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cfb10-5dde-46e0-895b-bc8d1cae66cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:03.829793 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cfb10-5dde-46e0-895b-bc8d1cae66cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:03.830469 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cfb10-5dde-46e0-895b-bc8d1cae66cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:03.832102 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cfb10-5dde-46e0-895b-bc8d1cae66cd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:03.842005 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1422/2852] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:03.846005 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad8ca6-1da6-4a08-95cc-4dd8b4a90c51 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:03.856031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad8ca6-1da6-4a08-95cc-4dd8b4a90c51 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:03.860603 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bebb70-8613-441f-8901-0a3964e809f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:03.862770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bebb70-8613-441f-8901-0a3964e809f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:03.862770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bebb70-8613-441f-8901-0a3964e809f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:03.862770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bebb70-8613-441f-8901-0a3964e809f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:03.862770 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bebb70-8613-441f-8901-0a3964e809f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:03.864021 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bebb70-8613-441f-8901-0a3964e809f7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:03.877087 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1431/2853] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:03.881560 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cfb10-5dde-46e0-895b-bc8d1cae66cd admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:03.892326 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cfb10-5dde-46e0-895b-bc8d1cae66cd admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:03.897014 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4324e-b21e-4f32-bb94-c13489c53a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:03.897688 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4324e-b21e-4f32-bb94-c13489c53a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:03.898151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4324e-b21e-4f32-bb94-c13489c53a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:03.898151 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4324e-b21e-4f32-bb94-c13489c53a84 admin admin] PATH_INFO: `/v3/projects/5940f6a39af44eee976b554a01360e51/users/60122e4f6d3d4a74b80c90253aa48df9/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:03.912687 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4324e-b21e-4f32-bb94-c13489c53a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5940f6a39af44eee976b554a01360e51, user_id=60122e4f6d3d4a74b80c90253aa48df9, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:03.915377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4324e-b21e-4f32-bb94-c13489c53a84 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:03.990355 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1423/2854] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:43:03 2022] PUT /identity/v3/projects/5940f6a39af44eee976b554a01360e51/users/60122e4f6d3d4a74b80c90253aa48df9/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:04.002635 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba1435-941d-4be4-b809-1e928c98b3e1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:04.002924 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba1435-941d-4be4-b809-1e928c98b3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:04.003200 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba1435-941d-4be4-b809-1e928c98b3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:04.028334 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:04.031400 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2bba1435-941d-4be4-b809-1e928c98b3e1 None None] MFA Rules not processed for user `60122e4f6d3d4a74b80c90253aa48df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:04.032626 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2bba1435-941d-4be4-b809-1e928c98b3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:04.097180 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1432/2855] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:04.102007 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4324e-b21e-4f32-bb94-c13489c53a84 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:04.165192 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4324e-b21e-4f32-bb94-c13489c53a84 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:04.170412 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ae012-6ca4-49cc-b75b-208e0626ad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:04.171015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ae012-6ca4-49cc-b75b-208e0626ad7b admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:04.171292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ae012-6ca4-49cc-b75b-208e0626ad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:04.171823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ae012-6ca4-49cc-b75b-208e0626ad7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:04.172218 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ae012-6ca4-49cc-b75b-208e0626ad7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:04.174156 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ae012-6ca4-49cc-b75b-208e0626ad7b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:04.288501 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1424/2856] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:43:04 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:43:04.292656 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba1435-941d-4be4-b809-1e928c98b3e1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:04.303496 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba1435-941d-4be4-b809-1e928c98b3e1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:04.365853 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e980f4-e6af-417f-a0b4-7dce99ca58a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:04.366420 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e980f4-e6af-417f-a0b4-7dce99ca58a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:04.366700 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e980f4-e6af-417f-a0b4-7dce99ca58a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:04.366994 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e980f4-e6af-417f-a0b4-7dce99ca58a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:04.367623 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e980f4-e6af-417f-a0b4-7dce99ca58a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:04.370133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e980f4-e6af-417f-a0b4-7dce99ca58a6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:04.468759 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1433/2857] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:43:04 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:43:04.472858 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ae012-6ca4-49cc-b75b-208e0626ad7b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:04.483605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ae012-6ca4-49cc-b75b-208e0626ad7b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:04.488403 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a9c6e-44da-453b-a3b3-24771e299fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:04.488992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a9c6e-44da-453b-a3b3-24771e299fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:04.489267 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a9c6e-44da-453b-a3b3-24771e299fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:04.489599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a9c6e-44da-453b-a3b3-24771e299fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:04.490292 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a9c6e-44da-453b-a3b3-24771e299fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:04.491878 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a9c6e-44da-453b-a3b3-24771e299fa2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:04.503748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1425/2858] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:04.508056 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e980f4-e6af-417f-a0b4-7dce99ca58a6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:04.519019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e980f4-e6af-417f-a0b4-7dce99ca58a6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:04.523794 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b783f8d-31a6-4cb7-8773-9bee562eb515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:04.524478 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b783f8d-31a6-4cb7-8773-9bee562eb515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:04.524761 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b783f8d-31a6-4cb7-8773-9bee562eb515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:04.525061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b783f8d-31a6-4cb7-8773-9bee562eb515 admin admin] PATH_INFO: `/v3/projects/0e30b33cb753463591ecc86675d026b8/users/0a775e060d3745c691a009b1220e4f0c/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:04.539768 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b783f8d-31a6-4cb7-8773-9bee562eb515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e30b33cb753463591ecc86675d026b8, user_id=0a775e060d3745c691a009b1220e4f0c, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:04.542492 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b783f8d-31a6-4cb7-8773-9bee562eb515 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:04.653112 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1434/2859] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:43:04 2022] PUT /identity/v3/projects/0e30b33cb753463591ecc86675d026b8/users/0a775e060d3745c691a009b1220e4f0c/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:04.656860 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a9c6e-44da-453b-a3b3-24771e299fa2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:04.670812 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a9c6e-44da-453b-a3b3-24771e299fa2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:04.733472 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c731d-0834-4f68-ac90-9891c58535f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:04.734127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c731d-0834-4f68-ac90-9891c58535f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:04.739058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c731d-0834-4f68-ac90-9891c58535f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:04.739058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c731d-0834-4f68-ac90-9891c58535f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:04.739058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c731d-0834-4f68-ac90-9891c58535f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:04.739058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c731d-0834-4f68-ac90-9891c58535f3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:04.747768 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1426/2860] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:04.752308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b783f8d-31a6-4cb7-8773-9bee562eb515 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:04.762842 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b783f8d-31a6-4cb7-8773-9bee562eb515 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:04.767598 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e72742-171b-4c3a-977a-86fd1d408f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:04.768245 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e72742-171b-4c3a-977a-86fd1d408f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:04.768527 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e72742-171b-4c3a-977a-86fd1d408f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:04.768804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e72742-171b-4c3a-977a-86fd1d408f74 admin admin] PATH_INFO: `/v3/projects/0e30b33cb753463591ecc86675d026b8/users/0a775e060d3745c691a009b1220e4f0c/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:04.770755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e72742-171b-4c3a-977a-86fd1d408f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e30b33cb753463591ecc86675d026b8, user_id=0a775e060d3745c691a009b1220e4f0c, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:04.773471 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e72742-171b-4c3a-977a-86fd1d408f74 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:04.784140 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1435/2861] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:43:04 2022] PUT /identity/v3/projects/0e30b33cb753463591ecc86675d026b8/users/0a775e060d3745c691a009b1220e4f0c/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:04.794854 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bba668-6437-4b91-863b-ad24a3704040 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:04.795165 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bba668-6437-4b91-863b-ad24a3704040 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:04.795465 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bba668-6437-4b91-863b-ad24a3704040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:04.823483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b1bba668-6437-4b91-863b-ad24a3704040 None None] MFA Rules not processed for user `0a775e060d3745c691a009b1220e4f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:04.824678 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b1bba668-6437-4b91-863b-ad24a3704040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:04.850328 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:04.892095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1427/2862] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:04.902837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e72742-171b-4c3a-977a-86fd1d408f74 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:04.904585 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-02e72742-171b-4c3a-977a-86fd1d408f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:04.962590 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e72742-171b-4c3a-977a-86fd1d408f74 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:04.966301 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea69578-0f3b-4e3d-964b-aefd23d32775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:04.966901 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea69578-0f3b-4e3d-964b-aefd23d32775 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:04.967184 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea69578-0f3b-4e3d-964b-aefd23d32775 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:04.967455 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea69578-0f3b-4e3d-964b-aefd23d32775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:04.975852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:05.029359 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea69578-0f3b-4e3d-964b-aefd23d32775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:05.030870 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea69578-0f3b-4e3d-964b-aefd23d32775 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:05.040063 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1436/2863] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:09.224145 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bba668-6437-4b91-863b-ad24a3704040 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:09.286605 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bba668-6437-4b91-863b-ad24a3704040 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:09.291242 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:09.291872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:09.292153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:09.292425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] PATH_INFO: `/v3/users/60122e4f6d3d4a74b80c90253aa48df9` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:09.293559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60122e4f6d3d4a74b80c90253aa48df9)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:09.295521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:09.502762 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60122e4f6d3d4a74b80c90253aa48df9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:09.503526 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60122e4f6d3d4a74b80c90253aa48df9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:09.504127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60122e4f6d3d4a74b80c90253aa48df9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:09.571263 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60122e4f6d3d4a74b80c90253aa48df9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:09.579337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60122e4f6d3d4a74b80c90253aa48df9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:09.602198 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1428/2864] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:43:09 2022] DELETE /identity/v3/users/60122e4f6d3d4a74b80c90253aa48df9 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:09.606936 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dd36c-db95-430f-bada-2ab17c98e423 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:09.607596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dd36c-db95-430f-bada-2ab17c98e423 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:09.607596 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dd36c-db95-430f-bada-2ab17c98e423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:09.628877 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7c7dd36c-db95-430f-bada-2ab17c98e423 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:09.630152 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7c7dd36c-db95-430f-bada-2ab17c98e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:09.691972 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1437/2865] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:09.702748 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:09.704139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:09.749622 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:09.762334 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37ad32-afbd-4674-9c1b-82f7a217ee41 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:09.765852 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250a154-9cab-4df3-a5ba-3097fd446510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:09.766485 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250a154-9cab-4df3-a5ba-3097fd446510 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:09.766758 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250a154-9cab-4df3-a5ba-3097fd446510 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:09.767028 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250a154-9cab-4df3-a5ba-3097fd446510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:09.776868 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250a154-9cab-4df3-a5ba-3097fd446510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:09.778425 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250a154-9cab-4df3-a5ba-3097fd446510 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:09.789990 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1429/2866] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:10.249733 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dd36c-db95-430f-bada-2ab17c98e423 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:10.253695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7c7dd36c-db95-430f-bada-2ab17c98e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:10.264264 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dd36c-db95-430f-bada-2ab17c98e423 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:10.269062 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:10.269725 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:10.270059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:10.270342 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] PATH_INFO: `/v3/projects/5940f6a39af44eee976b554a01360e51` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:10.271503 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5940f6a39af44eee976b554a01360e51)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:10.273484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:10.407829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5940f6a39af44eee976b554a01360e51', 'request_id': 'req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.413372 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5940f6a39af44eee976b554a01360e51', 'request_id': 'req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.528357 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5940f6a39af44eee976b554a01360e51', 'request_id': 'req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.541441 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] The token cache is being invalidate because project 5940f6a39af44eee976b554a01360e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:43:10.541806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.542495 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.545320 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1438/2867] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:43:10 2022] DELETE /identity/v3/projects/5940f6a39af44eee976b554a01360e51 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:10.549547 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250a154-9cab-4df3-a5ba-3097fd446510 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:10.559701 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250a154-9cab-4df3-a5ba-3097fd446510 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:10.615994 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:10.616543 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:10.616825 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:10.617103 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] PATH_INFO: `/v3/users/0a775e060d3745c691a009b1220e4f0c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:10.618271 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a775e060d3745c691a009b1220e4f0c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:10.620235 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:10.804172 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a775e060d3745c691a009b1220e4f0c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.804841 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a775e060d3745c691a009b1220e4f0c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.805454 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a775e060d3745c691a009b1220e4f0c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.870350 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a775e060d3745c691a009b1220e4f0c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.878012 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8ffa6c4d-2f87-4554-828d-ea1cb2427a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a775e060d3745c691a009b1220e4f0c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:10.895483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1430/2868] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:43:10 2022] DELETE /identity/v3/users/0a775e060d3745c691a009b1220e4f0c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:43:11.352175 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:11.354647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:11.368641 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d71ab-bebf-4950-a7bb-ad5b3eb994c1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:11.401570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:11.428051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:11.429282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:11.429282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:11.429282 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] PATH_INFO: `/v3/projects/0e30b33cb753463591ecc86675d026b8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:11.430500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e30b33cb753463591ecc86675d026b8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:11.432418 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:11.544514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e30b33cb753463591ecc86675d026b8', 'request_id': 'req-aa583cac-ddb0-4850-943e-b4e10d9c927b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:11.550558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e30b33cb753463591ecc86675d026b8', 'request_id': 'req-aa583cac-ddb0-4850-943e-b4e10d9c927b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:11.675318 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e30b33cb753463591ecc86675d026b8', 'request_id': 'req-aa583cac-ddb0-4850-943e-b4e10d9c927b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:11.687164 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] The token cache is being invalidate because project 0e30b33cb753463591ecc86675d026b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:43:11.687447 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:11.688104 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-aa583cac-ddb0-4850-943e-b4e10d9c927b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:11.689646 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1439/2869] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:43:11 2022] DELETE /identity/v3/projects/0e30b33cb753463591ecc86675d026b8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:11.694644 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24a96f-4a51-4ac9-825a-fe68e9b2e52b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:11.695135 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24a96f-4a51-4ac9-825a-fe68e9b2e52b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:11.695253 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24a96f-4a51-4ac9-825a-fe68e9b2e52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:11.715978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cf24a96f-4a51-4ac9-825a-fe68e9b2e52b None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:11.717197 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cf24a96f-4a51-4ac9-825a-fe68e9b2e52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:11.775583 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1431/2870] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:11.783506 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3fb0-2148-4ab8-bc31-8920ed73adb6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:11.783829 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3fb0-2148-4ab8-bc31-8920ed73adb6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:11.784109 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3fb0-2148-4ab8-bc31-8920ed73adb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:11.803433 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d35b3fb0-2148-4ab8-bc31-8920ed73adb6 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:11.804636 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d35b3fb0-2148-4ab8-bc31-8920ed73adb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:11.813035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1440/2871] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:11.817212 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24a96f-4a51-4ac9-825a-fe68e9b2e52b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:11.823842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:11.827833 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24a96f-4a51-4ac9-825a-fe68e9b2e52b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:11.832437 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4670c4-ec4c-419c-90e0-c0749797d9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:11.837402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4670c4-ec4c-419c-90e0-c0749797d9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:11.837402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4670c4-ec4c-419c-90e0-c0749797d9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:11.837402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4670c4-ec4c-419c-90e0-c0749797d9c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:11.837402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4670c4-ec4c-419c-90e0-c0749797d9c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:43:11.837402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4670c4-ec4c-419c-90e0-c0749797d9c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:11.837402 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4670c4-ec4c-419c-90e0-c0749797d9c0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:11.847719 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1432/2872] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:43:11.852103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3fb0-2148-4ab8-bc31-8920ed73adb6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:11.861519 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3fb0-2148-4ab8-bc31-8920ed73adb6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:11.866305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956ca4c-1641-4f47-baf8-48dcdb7d1b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:11.866900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956ca4c-1641-4f47-baf8-48dcdb7d1b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:11.867178 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956ca4c-1641-4f47-baf8-48dcdb7d1b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:11.867452 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956ca4c-1641-4f47-baf8-48dcdb7d1b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:11.868083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956ca4c-1641-4f47-baf8-48dcdb7d1b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:11.870551 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956ca4c-1641-4f47-baf8-48dcdb7d1b87 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:11.974410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1441/2873] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:43:11 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:43:11.979281 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4670c4-ec4c-419c-90e0-c0749797d9c0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:11.989615 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4670c4-ec4c-419c-90e0-c0749797d9c0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:12.046308 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d439b4-15ca-4d88-b4c1-948aea2a65a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:12.046909 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d439b4-15ca-4d88-b4c1-948aea2a65a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:12.047191 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d439b4-15ca-4d88-b4c1-948aea2a65a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:12.047466 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d439b4-15ca-4d88-b4c1-948aea2a65a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:12.048111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d439b4-15ca-4d88-b4c1-948aea2a65a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:12.050139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d439b4-15ca-4d88-b4c1-948aea2a65a3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:12.202435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1433/2874] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:43:11 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:43:12.206619 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956ca4c-1641-4f47-baf8-48dcdb7d1b87 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:12.217912 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956ca4c-1641-4f47-baf8-48dcdb7d1b87 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:12.222395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b5889-c4af-465e-8cef-25fdfe1d6386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:12.222969 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b5889-c4af-465e-8cef-25fdfe1d6386 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:12.223244 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b5889-c4af-465e-8cef-25fdfe1d6386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:12.223517 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b5889-c4af-465e-8cef-25fdfe1d6386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:12.224199 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b5889-c4af-465e-8cef-25fdfe1d6386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:12.225799 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b5889-c4af-465e-8cef-25fdfe1d6386 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:12.235758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1442/2875] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:12.239727 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d439b4-15ca-4d88-b4c1-948aea2a65a3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:12.250337 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d439b4-15ca-4d88-b4c1-948aea2a65a3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:12.255096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c784558-aaa9-4021-be67-a0a6051fd09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:12.255689 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c784558-aaa9-4021-be67-a0a6051fd09b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:12.255972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c784558-aaa9-4021-be67-a0a6051fd09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:12.256247 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c784558-aaa9-4021-be67-a0a6051fd09b admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:12.256921 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c784558-aaa9-4021-be67-a0a6051fd09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:12.258545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c784558-aaa9-4021-be67-a0a6051fd09b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:12.269576 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1434/2876] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:12.273622 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b5889-c4af-465e-8cef-25fdfe1d6386 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:12.284398 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b5889-c4af-465e-8cef-25fdfe1d6386 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:12.289046 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18c12e-b080-4381-972f-1e5a4213f075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:12.289698 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18c12e-b080-4381-972f-1e5a4213f075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:12.290008 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18c12e-b080-4381-972f-1e5a4213f075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:12.290283 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18c12e-b080-4381-972f-1e5a4213f075 admin admin] PATH_INFO: `/v3/projects/977ff9dafaf34395a90731ce280ca55e/users/1ff6d324f933415298ac7dfff7fd62a2/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:12.305525 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18c12e-b080-4381-972f-1e5a4213f075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=977ff9dafaf34395a90731ce280ca55e, user_id=1ff6d324f933415298ac7dfff7fd62a2, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:12.308851 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18c12e-b080-4381-972f-1e5a4213f075 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:12.394269 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1443/2877] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:43:12 2022] PUT /identity/v3/projects/977ff9dafaf34395a90731ce280ca55e/users/1ff6d324f933415298ac7dfff7fd62a2/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:12.405598 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4043a6-38ea-4d7d-b8ea-7ce7d2d3c787 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:12.405895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4043a6-38ea-4d7d-b8ea-7ce7d2d3c787 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:12.406181 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4043a6-38ea-4d7d-b8ea-7ce7d2d3c787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:12.437607 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fb4043a6-38ea-4d7d-b8ea-7ce7d2d3c787 None None] MFA Rules not processed for user `1ff6d324f933415298ac7dfff7fd62a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:12.438842 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fb4043a6-38ea-4d7d-b8ea-7ce7d2d3c787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:12.502704 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1435/2878] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:12.515014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18c12e-b080-4381-972f-1e5a4213f075 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:12.516729 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ef18c12e-b080-4381-972f-1e5a4213f075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:12.577442 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18c12e-b080-4381-972f-1e5a4213f075 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:12.581322 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1aa26-17a2-41c2-9086-ff1915360b32 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f214c2b77d4cbab2f15122ba21bf38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:12.581960 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1aa26-17a2-41c2-9086-ff1915360b32 service glance] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:12.582242 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1aa26-17a2-41c2-9086-ff1915360b32 service glance] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:12.582514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1aa26-17a2-41c2-9086-ff1915360b32 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:12.591248 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1aa26-17a2-41c2-9086-ff1915360b32 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:12.592689 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1aa26-17a2-41c2-9086-ff1915360b32 service glance] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:12.601735 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1444/2879] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:22.499618 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:22.597633 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:48.171167 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4043a6-38ea-4d7d-b8ea-7ce7d2d3c787 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:48.250996 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4043a6-38ea-4d7d-b8ea-7ce7d2d3c787 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:48.256797 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:48.257453 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:48.257731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:48.258033 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] PATH_INFO: `/v3/users/1ff6d324f933415298ac7dfff7fd62a2` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:48.259659 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff6d324f933415298ac7dfff7fd62a2)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:48.261693 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:48.488665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff6d324f933415298ac7dfff7fd62a2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:48.489342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff6d324f933415298ac7dfff7fd62a2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:48.489982 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff6d324f933415298ac7dfff7fd62a2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:48.566300 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff6d324f933415298ac7dfff7fd62a2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:48.574655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff6d324f933415298ac7dfff7fd62a2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:48.592663 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1436/2880] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:43:48 2022] DELETE /identity/v3/users/1ff6d324f933415298ac7dfff7fd62a2 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:48.597542 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ece09-dfdc-4140-ab19-f37977cbc563 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:48.598107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ece09-dfdc-4140-ab19-f37977cbc563 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:48.598296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ece09-dfdc-4140-ab19-f37977cbc563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:48.640600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-110ece09-dfdc-4140-ab19-f37977cbc563 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:48.642560 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-110ece09-dfdc-4140-ab19-f37977cbc563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:48.700028 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1445/2881] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:48.709818 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:48.711279 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:48.782261 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:48.791168 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a644ff0-6dd9-4109-bb6a-0690471d6543 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:48.797139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac6c28-2828-4cdb-8030-35513eba2b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:48.797139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac6c28-2828-4cdb-8030-35513eba2b02 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:48.797139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac6c28-2828-4cdb-8030-35513eba2b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:48.797139 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac6c28-2828-4cdb-8030-35513eba2b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:48.805015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac6c28-2828-4cdb-8030-35513eba2b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:48.806981 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac6c28-2828-4cdb-8030-35513eba2b02 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:48.816445 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1437/2882] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:49.218034 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ece09-dfdc-4140-ab19-f37977cbc563 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:49.219494 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-110ece09-dfdc-4140-ab19-f37977cbc563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:49.232023 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ece09-dfdc-4140-ab19-f37977cbc563 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:49.237007 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:49.237639 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.237945 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.238217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] PATH_INFO: `/v3/projects/977ff9dafaf34395a90731ce280ca55e` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.239410 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=977ff9dafaf34395a90731ce280ca55e)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:49.241336 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:49.345232 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977ff9dafaf34395a90731ce280ca55e', 'request_id': 'req-91c10b2c-325d-4112-9714-5acc6c704f29'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:49.351369 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977ff9dafaf34395a90731ce280ca55e', 'request_id': 'req-91c10b2c-325d-4112-9714-5acc6c704f29'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:49.423297 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977ff9dafaf34395a90731ce280ca55e', 'request_id': 'req-91c10b2c-325d-4112-9714-5acc6c704f29'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:49.436378 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] The token cache is being invalidate because project 977ff9dafaf34395a90731ce280ca55e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:43:49.436688 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:49.437360 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c10b2c-325d-4112-9714-5acc6c704f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:43:49.438895 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1446/2883] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:43:49 2022] DELETE /identity/v3/projects/977ff9dafaf34395a90731ce280ca55e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:49.443836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7005f3c-7d02-4d0a-ac1b-8fdb388bc81d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.453398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7005f3c-7d02-4d0a-ac1b-8fdb388bc81d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.453398 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7005f3c-7d02-4d0a-ac1b-8fdb388bc81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.483636 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d7005f3c-7d02-4d0a-ac1b-8fdb388bc81d None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:49.484859 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d7005f3c-7d02-4d0a-ac1b-8fdb388bc81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:49.546630 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1438/2884] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:49.554518 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bf39f-9efd-47a2-9704-f31e03cff98c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.554887 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bf39f-9efd-47a2-9704-f31e03cff98c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.555682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bf39f-9efd-47a2-9704-f31e03cff98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.576168 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-cb1bf39f-9efd-47a2-9704-f31e03cff98c None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:49.577462 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cb1bf39f-9efd-47a2-9704-f31e03cff98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:49.585514 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1447/2885] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:49.590008 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7005f3c-7d02-4d0a-ac1b-8fdb388bc81d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:49.600021 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7005f3c-7d02-4d0a-ac1b-8fdb388bc81d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:49.604690 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e094e52-a916-41e3-8131-6f2c8ab0b0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:49.605344 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e094e52-a916-41e3-8131-6f2c8ab0b0ba admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.605619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e094e52-a916-41e3-8131-6f2c8ab0b0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.605913 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e094e52-a916-41e3-8131-6f2c8ab0b0ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.606569 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e094e52-a916-41e3-8131-6f2c8ab0b0ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:43:49.606866 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e094e52-a916-41e3-8131-6f2c8ab0b0ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:49.609233 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e094e52-a916-41e3-8131-6f2c8ab0b0ba admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:49.623872 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1439/2886] 10.222.0.83 () {58 vars in 1153 bytes} [Wed Aug 24 11:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:43:49.628655 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bf39f-9efd-47a2-9704-f31e03cff98c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:49.640143 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bf39f-9efd-47a2-9704-f31e03cff98c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:49.645223 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741974c-790f-4b51-afef-50eb4594ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:49.645784 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741974c-790f-4b51-afef-50eb4594ef02 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.646103 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741974c-790f-4b51-afef-50eb4594ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.646377 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741974c-790f-4b51-afef-50eb4594ef02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.647015 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741974c-790f-4b51-afef-50eb4594ef02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:49.648915 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741974c-790f-4b51-afef-50eb4594ef02 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:49.739211 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1448/2887] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:43:49 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:43:49.744096 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e094e52-a916-41e3-8131-6f2c8ab0b0ba admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:49.754515 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e094e52-a916-41e3-8131-6f2c8ab0b0ba admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:49.810907 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74c55b-536f-48e0-964e-15190720bee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:49.811504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74c55b-536f-48e0-964e-15190720bee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.811786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74c55b-536f-48e0-964e-15190720bee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.812068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74c55b-536f-48e0-964e-15190720bee8 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.812700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74c55b-536f-48e0-964e-15190720bee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:49.814661 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74c55b-536f-48e0-964e-15190720bee8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:49.880643 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1440/2888] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:43:49 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:43:49.891296 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741974c-790f-4b51-afef-50eb4594ef02 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:49.902680 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741974c-790f-4b51-afef-50eb4594ef02 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:49.907262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8dcc21-09ca-4de3-b76f-e22cff4b543b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:49.907855 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8dcc21-09ca-4de3-b76f-e22cff4b543b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.908133 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8dcc21-09ca-4de3-b76f-e22cff4b543b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.908405 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8dcc21-09ca-4de3-b76f-e22cff4b543b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.909051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8dcc21-09ca-4de3-b76f-e22cff4b543b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:49.910730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8dcc21-09ca-4de3-b76f-e22cff4b543b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:49.920946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1449/2889] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:49.935994 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74c55b-536f-48e0-964e-15190720bee8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:49.946895 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74c55b-536f-48e0-964e-15190720bee8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:49.951566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363e16c-b56a-452e-a180-46a2e58297c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:49.952163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363e16c-b56a-452e-a180-46a2e58297c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.952451 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363e16c-b56a-452e-a180-46a2e58297c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.952731 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363e16c-b56a-452e-a180-46a2e58297c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.953452 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363e16c-b56a-452e-a180-46a2e58297c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:49.955077 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363e16c-b56a-452e-a180-46a2e58297c1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:49.966545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1441/2890] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:49.970738 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8dcc21-09ca-4de3-b76f-e22cff4b543b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:49.981314 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8dcc21-09ca-4de3-b76f-e22cff4b543b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:49.988331 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440038f-bae2-4611-9303-e93e953f7aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:49.988992 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440038f-bae2-4611-9303-e93e953f7aca admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.989262 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440038f-bae2-4611-9303-e93e953f7aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.989591 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440038f-bae2-4611-9303-e93e953f7aca admin admin] PATH_INFO: `/v3/projects/548b5e4f59b24f0a9b88bf125c8a19a8/users/4edfa99b10014e8289697a255d8b90cf/roles/799b2f3554434281bf10b2602fbbc09a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:50.005889 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440038f-bae2-4611-9303-e93e953f7aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=548b5e4f59b24f0a9b88bf125c8a19a8, user_id=4edfa99b10014e8289697a255d8b90cf, role_id=799b2f3554434281bf10b2602fbbc09a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:50.009170 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440038f-bae2-4611-9303-e93e953f7aca admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:50.114716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1450/2891] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:43:49 2022] PUT /identity/v3/projects/548b5e4f59b24f0a9b88bf125c8a19a8/users/4edfa99b10014e8289697a255d8b90cf/roles/799b2f3554434281bf10b2602fbbc09a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:43:50.120532 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6beda0d-b0c4-4d35-872f-eee2c1de0115 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:50.120823 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6beda0d-b0c4-4d35-872f-eee2c1de0115 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:50.121161 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6beda0d-b0c4-4d35-872f-eee2c1de0115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:50.141956 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c6beda0d-b0c4-4d35-872f-eee2c1de0115 None None] MFA Rules not processed for user `5d32a6d718b64e35aca00f733d966fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:50.145365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c6beda0d-b0c4-4d35-872f-eee2c1de0115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:50.173599 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:50.203330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1442/2892] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:50.212730 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440038f-bae2-4611-9303-e93e953f7aca admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:50.214600 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6440038f-bae2-4611-9303-e93e953f7aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:50.220079 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:50.273570 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440038f-bae2-4611-9303-e93e953f7aca admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:50.277278 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759c513-1db1-4d9f-8d73-47fc0c05de47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:50.277899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759c513-1db1-4d9f-8d73-47fc0c05de47 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:50.278191 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759c513-1db1-4d9f-8d73-47fc0c05de47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:50.278463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759c513-1db1-4d9f-8d73-47fc0c05de47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:50.288075 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759c513-1db1-4d9f-8d73-47fc0c05de47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:50.289539 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759c513-1db1-4d9f-8d73-47fc0c05de47 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:50.300029 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1451/2893] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:55.824342 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7714f-e402-4379-a107-eef29eb01604 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:55.824899 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7714f-e402-4379-a107-eef29eb01604 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:55.825134 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7714f-e402-4379-a107-eef29eb01604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:55.859686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cca7714f-e402-4379-a107-eef29eb01604 None None] MFA Rules not processed for user `4edfa99b10014e8289697a255d8b90cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:55.860936 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cca7714f-e402-4379-a107-eef29eb01604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:55.946173 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1443/2894] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:43:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:55.949577 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759c513-1db1-4d9f-8d73-47fc0c05de47 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:55.960107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759c513-1db1-4d9f-8d73-47fc0c05de47 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:55.966315 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15555f9a-7c33-44bb-8e40-74a1713051dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:55.966937 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15555f9a-7c33-44bb-8e40-74a1713051dc admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:55.967221 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15555f9a-7c33-44bb-8e40-74a1713051dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:55.967497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15555f9a-7c33-44bb-8e40-74a1713051dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:55.968150 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15555f9a-7c33-44bb-8e40-74a1713051dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:55.970608 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15555f9a-7c33-44bb-8e40-74a1713051dc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:56.073355 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1452/2895] 10.222.0.83 () {60 vars in 1153 bytes} [Wed Aug 24 11:43:55 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:43:56.077755 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7714f-e402-4379-a107-eef29eb01604 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:56.089638 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7714f-e402-4379-a107-eef29eb01604 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:56.155206 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa223b8f-69f8-4b3b-9fba-318b64621ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:56.155824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa223b8f-69f8-4b3b-9fba-318b64621ede admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:56.156104 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa223b8f-69f8-4b3b-9fba-318b64621ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:56.156377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa223b8f-69f8-4b3b-9fba-318b64621ede admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:56.157020 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa223b8f-69f8-4b3b-9fba-318b64621ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:56.159128 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa223b8f-69f8-4b3b-9fba-318b64621ede admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:56.274904 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1444/2896] 10.222.0.83 () {60 vars in 1144 bytes} [Wed Aug 24 11:43:56 2022] POST /identity/v3/users => generated 295 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:43:56.278932 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15555f9a-7c33-44bb-8e40-74a1713051dc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:56.290335 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15555f9a-7c33-44bb-8e40-74a1713051dc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:56.295149 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb94667-c800-457c-95b1-8e0ad8ed8a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:56.295781 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb94667-c800-457c-95b1-8e0ad8ed8a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:56.296061 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb94667-c800-457c-95b1-8e0ad8ed8a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:56.296337 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb94667-c800-457c-95b1-8e0ad8ed8a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:56.297020 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb94667-c800-457c-95b1-8e0ad8ed8a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:56.298827 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb94667-c800-457c-95b1-8e0ad8ed8a17 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:56.311230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1453/2897] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:56.315752 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa223b8f-69f8-4b3b-9fba-318b64621ede admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:56.327461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa223b8f-69f8-4b3b-9fba-318b64621ede admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:56.332965 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32848878-0453-4e89-ab8c-5a6648b9c6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:56.334023 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32848878-0453-4e89-ab8c-5a6648b9c6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:56.334327 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32848878-0453-4e89-ab8c-5a6648b9c6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:56.334592 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32848878-0453-4e89-ab8c-5a6648b9c6cd admin admin] PATH_INFO: `/v3/projects/77647159ba75424ab303d44de5e7eef7/users/79df3a80ae684517b81ddedc1de0a949/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:56.351180 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32848878-0453-4e89-ab8c-5a6648b9c6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=77647159ba75424ab303d44de5e7eef7, user_id=79df3a80ae684517b81ddedc1de0a949, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:56.354566 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32848878-0453-4e89-ab8c-5a6648b9c6cd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:56.492015 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1445/2898] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:43:56 2022] PUT /identity/v3/projects/77647159ba75424ab303d44de5e7eef7/users/79df3a80ae684517b81ddedc1de0a949/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:43:56.496267 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb94667-c800-457c-95b1-8e0ad8ed8a17 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:56.508500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb94667-c800-457c-95b1-8e0ad8ed8a17 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:56.567190 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0509631-7ea1-484a-b008-241ca9603ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:56.567885 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0509631-7ea1-484a-b008-241ca9603ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:56.568230 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0509631-7ea1-484a-b008-241ca9603ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:56.568567 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0509631-7ea1-484a-b008-241ca9603ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:56.569308 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0509631-7ea1-484a-b008-241ca9603ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:56.571500 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0509631-7ea1-484a-b008-241ca9603ec4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:56.581756 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1454/2899] 10.222.0.83 () {58 vars in 1122 bytes} [Wed Aug 24 11:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:43:56.586928 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32848878-0453-4e89-ab8c-5a6648b9c6cd admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:56.597855 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32848878-0453-4e89-ab8c-5a6648b9c6cd admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:56.602577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95335cc3-f65c-41a0-a2ee-929b451181d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:56.603231 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95335cc3-f65c-41a0-a2ee-929b451181d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:56.603512 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95335cc3-f65c-41a0-a2ee-929b451181d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:56.603786 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95335cc3-f65c-41a0-a2ee-929b451181d5 admin admin] PATH_INFO: `/v3/projects/77647159ba75424ab303d44de5e7eef7/users/79df3a80ae684517b81ddedc1de0a949/roles/80b00b8da5b24da8bce7c6fe885050f8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:56.606059 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95335cc3-f65c-41a0-a2ee-929b451181d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77647159ba75424ab303d44de5e7eef7, user_id=79df3a80ae684517b81ddedc1de0a949, role_id=80b00b8da5b24da8bce7c6fe885050f8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:56.608721 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95335cc3-f65c-41a0-a2ee-929b451181d5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:56.620323 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1446/2900] 10.222.0.83 () {60 vars in 1483 bytes} [Wed Aug 24 11:43:56 2022] PUT /identity/v3/projects/77647159ba75424ab303d44de5e7eef7/users/79df3a80ae684517b81ddedc1de0a949/roles/80b00b8da5b24da8bce7c6fe885050f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:44:02.328438 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0509631-7ea1-484a-b008-241ca9603ec4 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:02.328942 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28335d-5524-46bf-9df3-0c27602c64e5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:02.329335 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28335d-5524-46bf-9df3-0c27602c64e5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:02.329458 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28335d-5524-46bf-9df3-0c27602c64e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:02.331982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f0509631-7ea1-484a-b008-241ca9603ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:02.364221 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2f28335d-5524-46bf-9df3-0c27602c64e5 None None] MFA Rules not processed for user `79df3a80ae684517b81ddedc1de0a949`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:44:02.365475 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2f28335d-5524-46bf-9df3-0c27602c64e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:02.374688 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:02.423631 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:02.448750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0509631-7ea1-484a-b008-241ca9603ec4 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:02.457226 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1447/2901] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:44:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:44:02.474238 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28335d-5524-46bf-9df3-0c27602c64e5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:02.476022 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2f28335d-5524-46bf-9df3-0c27602c64e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:02.512334 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2ef52-b83c-46ae-9f16-3ff109085616 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:02.513059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2ef52-b83c-46ae-9f16-3ff109085616 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:02.513489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2ef52-b83c-46ae-9f16-3ff109085616 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:02.513820 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2ef52-b83c-46ae-9f16-3ff109085616 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:02.515558 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-71f2ef52-b83c-46ae-9f16-3ff109085616 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:02.528395 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2ef52-b83c-46ae-9f16-3ff109085616 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:02.530142 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2ef52-b83c-46ae-9f16-3ff109085616 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:02.542083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1455/2902] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:44:02.594973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28335d-5524-46bf-9df3-0c27602c64e5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:02.594973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ae93-87f1-477e-98f3-28a19efa2746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:02.594973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ae93-87f1-477e-98f3-28a19efa2746 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:02.594973 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ae93-87f1-477e-98f3-28a19efa2746 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:02.595393 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ae93-87f1-477e-98f3-28a19efa2746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:02.653545 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10ae93-87f1-477e-98f3-28a19efa2746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:02.655196 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10ae93-87f1-477e-98f3-28a19efa2746 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:02.665038 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1448/2903] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:44:12.536962 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=36 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:12.660111 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:27.323909 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a97cc-82e7-4785-83a6-27e763575640 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:27.324456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a97cc-82e7-4785-83a6-27e763575640 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:27.324456 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a97cc-82e7-4785-83a6-27e763575640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:27.343119 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c16a97cc-82e7-4785-83a6-27e763575640 None None] MFA Rules not processed for user `4edfa99b10014e8289697a255d8b90cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:44:27.344323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c16a97cc-82e7-4785-83a6-27e763575640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:27.351826 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1456/2904] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:44:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:44:27.666347 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ae93-87f1-477e-98f3-28a19efa2746 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:27.720188 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ae93-87f1-477e-98f3-28a19efa2746 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:27.723821 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d12d14-a0de-48e6-90a7-302512f97be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:27.724419 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d12d14-a0de-48e6-90a7-302512f97be2 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:27.724700 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d12d14-a0de-48e6-90a7-302512f97be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:27.725032 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d12d14-a0de-48e6-90a7-302512f97be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:27.727278 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-50d12d14-a0de-48e6-90a7-302512f97be2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:27.737068 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d12d14-a0de-48e6-90a7-302512f97be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:27.738533 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d12d14-a0de-48e6-90a7-302512f97be2 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:27.747224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1449/2905] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:44:29.421219 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a97cc-82e7-4785-83a6-27e763575640 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:29.422682 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c16a97cc-82e7-4785-83a6-27e763575640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:29.437661 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a97cc-82e7-4785-83a6-27e763575640 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:29.442755 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:29.443358 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:29.443640 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:29.443910 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:29.446243 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:29.457852 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:29.459463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:29.470611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1457/2906] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:44:36.529997 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d12d14-a0de-48e6-90a7-302512f97be2 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:36.548849 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d12d14-a0de-48e6-90a7-302512f97be2 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:36.554972 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:36.555589 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:36.555874 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:36.556413 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:36.558060 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:36.571574 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:36.579537 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:36.584378 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1450/2907] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:44:39.466059 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=9 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:46.579483 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=32 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:48.313750 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:48.315261 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:48.330393 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e2ca5-4ab9-4d6f-98da-c656c1f15382 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:48.336193 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf69e96-50b8-490b-b96a-baa7f7287903 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:48.336801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf69e96-50b8-490b-b96a-baa7f7287903 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:48.337083 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf69e96-50b8-490b-b96a-baa7f7287903 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:48.337387 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf69e96-50b8-490b-b96a-baa7f7287903 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:48.339182 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ddf69e96-50b8-490b-b96a-baa7f7287903 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:48.352181 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf69e96-50b8-490b-b96a-baa7f7287903 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:48.353773 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf69e96-50b8-490b-b96a-baa7f7287903 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:48.364716 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1458/2908] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:44:51.674130 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:51.675540 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:51.687672 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92117b-cff1-4b16-a092-3205e119b910 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:51.691380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d3011-c6ef-471d-926d-262ec8e37ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:51.691912 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d3011-c6ef-471d-926d-262ec8e37ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:51.692186 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d3011-c6ef-471d-926d-262ec8e37ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:51.692461 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d3011-c6ef-471d-926d-262ec8e37ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:51.702208 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d3011-c6ef-471d-926d-262ec8e37ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:51.703629 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d3011-c6ef-471d-926d-262ec8e37ba7 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:51.713097 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1451/2909] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:44:58.360121 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:01.708835 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:06.105105 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf69e96-50b8-490b-b96a-baa7f7287903 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:06.115125 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf69e96-50b8-490b-b96a-baa7f7287903 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:06.118758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7403416-addf-41de-a3b6-337339078719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:06.119347 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7403416-addf-41de-a3b6-337339078719 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:06.119630 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7403416-addf-41de-a3b6-337339078719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:06.119899 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7403416-addf-41de-a3b6-337339078719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:06.121943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e7403416-addf-41de-a3b6-337339078719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:06.131859 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7403416-addf-41de-a3b6-337339078719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:06.133284 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7403416-addf-41de-a3b6-337339078719 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:06.141419 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1459/2910] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:16.137792 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:11.177154 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d3011-c6ef-471d-926d-262ec8e37ba7 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:11.186209 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d3011-c6ef-471d-926d-262ec8e37ba7 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:11.189863 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c9f75-dc59-4ae9-bd51-d0bbe5a71db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:11.190481 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c9f75-dc59-4ae9-bd51-d0bbe5a71db7 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:11.190767 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c9f75-dc59-4ae9-bd51-d0bbe5a71db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:11.191040 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c9f75-dc59-4ae9-bd51-d0bbe5a71db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:11.192773 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b99c9f75-dc59-4ae9-bd51-d0bbe5a71db7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:11.207559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c9f75-dc59-4ae9-bd51-d0bbe5a71db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:11.209017 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c9f75-dc59-4ae9-bd51-d0bbe5a71db7 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:11.220468 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1452/2911] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:12.403913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7403416-addf-41de-a3b6-337339078719 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:12.413804 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7403416-addf-41de-a3b6-337339078719 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:12.417695 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9aad0-e9b4-4eb8-9034-b15908a828f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:12.418343 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9aad0-e9b4-4eb8-9034-b15908a828f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:12.418620 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9aad0-e9b4-4eb8-9034-b15908a828f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:12.418890 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9aad0-e9b4-4eb8-9034-b15908a828f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:12.420652 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4fc9aad0-e9b4-4eb8-9034-b15908a828f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:12.435305 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9aad0-e9b4-4eb8-9034-b15908a828f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:12.436746 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9aad0-e9b4-4eb8-9034-b15908a828f4 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:12.448309 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1460/2912] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:17.103624 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b112d-2ad9-4cc3-87c3-33d2785d05e6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:17.104125 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b112d-2ad9-4cc3-87c3-33d2785d05e6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:17.104240 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b112d-2ad9-4cc3-87c3-33d2785d05e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:17.123380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1d8b112d-2ad9-4cc3-87c3-33d2785d05e6 None None] MFA Rules not processed for user `4edfa99b10014e8289697a255d8b90cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:17.124594 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1d8b112d-2ad9-4cc3-87c3-33d2785d05e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:17.131991 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1453/2913] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:47:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:22.443327 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:27.117665 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=9 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:11.336217 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9aad0-e9b4-4eb8-9034-b15908a828f4 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:11.347480 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9aad0-e9b4-4eb8-9034-b15908a828f4 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:11.353312 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f8762-d613-45f0-b0e4-7abc43b73905 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:11.353946 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f8762-d613-45f0-b0e4-7abc43b73905 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:11.354228 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f8762-d613-45f0-b0e4-7abc43b73905 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:11.354511 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f8762-d613-45f0-b0e4-7abc43b73905 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:11.363047 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f8762-d613-45f0-b0e4-7abc43b73905 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:11.364611 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f8762-d613-45f0-b0e4-7abc43b73905 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:11.375913 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1461/2914] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:49:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:49:21.371463 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:30.862836 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b112d-2ad9-4cc3-87c3-33d2785d05e6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:30.864214 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1d8b112d-2ad9-4cc3-87c3-33d2785d05e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:30.946484 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b112d-2ad9-4cc3-87c3-33d2785d05e6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:30.950256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d739b-6c6a-4a02-9474-dbff3cbd8521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9dd6f187dd84f04866da64732b242d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:30.950861 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d739b-6c6a-4a02-9474-dbff3cbd8521 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:30.951142 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d739b-6c6a-4a02-9474-dbff3cbd8521 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:30.951416 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d739b-6c6a-4a02-9474-dbff3cbd8521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:30.960655 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d739b-6c6a-4a02-9474-dbff3cbd8521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:30.962123 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d739b-6c6a-4a02-9474-dbff3cbd8521 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:30.971148 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1454/2915] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:49:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:49:40.967076 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=35 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:42.714816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a4d6-b380-4751-a00a-018dd422cb8f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:42.715367 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a4d6-b380-4751-a00a-018dd422cb8f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:42.715497 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a4d6-b380-4751-a00a-018dd422cb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:42.734628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4126a4d6-b380-4751-a00a-018dd422cb8f None None] MFA Rules not processed for user `4edfa99b10014e8289697a255d8b90cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:42.735836 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4126a4d6-b380-4751-a00a-018dd422cb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:42.743573 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1462/2916] 10.222.0.83 () {58 vars in 958 bytes} [Wed Aug 24 11:49:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:43.064002 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d739b-6c6a-4a02-9474-dbff3cbd8521 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:43.073257 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d739b-6c6a-4a02-9474-dbff3cbd8521 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:43.076867 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f5962-0f04-4f07-9c88-64c64fae999c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:43.077450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f5962-0f04-4f07-9c88-64c64fae999c service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:43.077723 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f5962-0f04-4f07-9c88-64c64fae999c service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:43.078043 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f5962-0f04-4f07-9c88-64c64fae999c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:43.079749 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-153f5962-0f04-4f07-9c88-64c64fae999c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:43.089800 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f5962-0f04-4f07-9c88-64c64fae999c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:43.091239 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f5962-0f04-4f07-9c88-64c64fae999c service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:43.100153 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1455/2917] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:49:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:49:45.875185 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126a4d6-b380-4751-a00a-018dd422cb8f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:45.887443 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126a4d6-b380-4751-a00a-018dd422cb8f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:45.892413 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50949c95-b54c-4f03-9f7c-ffbffaa8c4f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:45.893012 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50949c95-b54c-4f03-9f7c-ffbffaa8c4f7 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:45.893292 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50949c95-b54c-4f03-9f7c-ffbffaa8c4f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:45.893565 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50949c95-b54c-4f03-9f7c-ffbffaa8c4f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:45.902982 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50949c95-b54c-4f03-9f7c-ffbffaa8c4f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:45.904568 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50949c95-b54c-4f03-9f7c-ffbffaa8c4f7 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:45.915155 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1463/2918] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:49:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:49:52.982938 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f5962-0f04-4f07-9c88-64c64fae999c service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:52.998129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f5962-0f04-4f07-9c88-64c64fae999c service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:53.005715 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea777a-c72e-4309-b068-8851afebdc5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:53.012978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea777a-c72e-4309-b068-8851afebdc5b service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:53.012978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea777a-c72e-4309-b068-8851afebdc5b service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:53.012978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea777a-c72e-4309-b068-8851afebdc5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:53.017383 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea777a-c72e-4309-b068-8851afebdc5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:53.019129 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea777a-c72e-4309-b068-8851afebdc5b service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:53.031815 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1456/2919] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:49:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:49:55.910787 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=9 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:50:03.026058 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=12 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:50:04.047763 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50949c95-b54c-4f03-9f7c-ffbffaa8c4f7 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:50:04.060489 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50949c95-b54c-4f03-9f7c-ffbffaa8c4f7 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:50:04.066071 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e555ad5-ddec-411d-8e3e-e3adc451977d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:50:04.066663 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e555ad5-ddec-411d-8e3e-e3adc451977d service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:50:04.066943 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e555ad5-ddec-411d-8e3e-e3adc451977d service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:50:04.067218 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e555ad5-ddec-411d-8e3e-e3adc451977d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:50:04.076352 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e555ad5-ddec-411d-8e3e-e3adc451977d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:50:04.077996 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e555ad5-ddec-411d-8e3e-e3adc451977d service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:50:04.089678 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1464/2920] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:50:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:50:07.015215 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea777a-c72e-4309-b068-8851afebdc5b service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:50:07.025388 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea777a-c72e-4309-b068-8851afebdc5b service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:50:07.029095 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02877a-62d0-4fd4-a2db-507dee8e06aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:50:07.029686 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02877a-62d0-4fd4-a2db-507dee8e06aa service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:50:07.029992 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02877a-62d0-4fd4-a2db-507dee8e06aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:50:07.030268 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02877a-62d0-4fd4-a2db-507dee8e06aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:50:07.040066 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02877a-62d0-4fd4-a2db-507dee8e06aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:50:07.041489 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02877a-62d0-4fd4-a2db-507dee8e06aa service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:50:07.051113 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1457/2921] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:50:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:50:09.121855 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e555ad5-ddec-411d-8e3e-e3adc451977d service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:50:09.135082 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e555ad5-ddec-411d-8e3e-e3adc451977d service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:50:09.148350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d642ac1-183c-4a1f-a9f3-03ede7f305a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a28f9fb3134f33819e53e0c72c63b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:50:09.148350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d642ac1-183c-4a1f-a9f3-03ede7f305a3 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:50:09.148350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d642ac1-183c-4a1f-a9f3-03ede7f305a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:50:09.148350 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d642ac1-183c-4a1f-a9f3-03ede7f305a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:50:09.149809 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6d642ac1-183c-4a1f-a9f3-03ede7f305a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:50:09.163144 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d642ac1-183c-4a1f-a9f3-03ede7f305a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:50:09.164900 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d642ac1-183c-4a1f-a9f3-03ede7f305a3 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:50:09.177323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1465/2922] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:50:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:50:17.046851 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:50:19.172769 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=12 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:51:09.044055 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02877a-62d0-4fd4-a2db-507dee8e06aa service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:51:09.055964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02877a-62d0-4fd4-a2db-507dee8e06aa service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:51:09.061118 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a3d0b-7e36-42e3-b650-5fb960cb0a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:51:09.061753 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a3d0b-7e36-42e3-b650-5fb960cb0a4c service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:51:09.062147 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a3d0b-7e36-42e3-b650-5fb960cb0a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:51:09.062435 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a3d0b-7e36-42e3-b650-5fb960cb0a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:51:09.071519 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a3d0b-7e36-42e3-b650-5fb960cb0a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:51:09.072940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a3d0b-7e36-42e3-b650-5fb960cb0a4c service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:51:09.081504 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1458/2923] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:51:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:51:19.077668 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:52:13.025647 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d642ac1-183c-4a1f-a9f3-03ede7f305a3 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:13.035467 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d642ac1-183c-4a1f-a9f3-03ede7f305a3 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:13.039124 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69fd93-3b9f-48b3-a977-3fa7a090f563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:13.039720 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69fd93-3b9f-48b3-a977-3fa7a090f563 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:13.039991 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69fd93-3b9f-48b3-a977-3fa7a090f563 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:13.040258 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69fd93-3b9f-48b3-a977-3fa7a090f563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:13.051038 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69fd93-3b9f-48b3-a977-3fa7a090f563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:13.052484 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69fd93-3b9f-48b3-a977-3fa7a090f563 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:13.065019 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1466/2924] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:52:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:52:14.262978 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a3d0b-7e36-42e3-b650-5fb960cb0a4c service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:14.273126 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a3d0b-7e36-42e3-b650-5fb960cb0a4c service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:14.286380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a85784-baba-47fd-bcde-5d62a7343b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:14.286380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a85784-baba-47fd-bcde-5d62a7343b88 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:14.286380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a85784-baba-47fd-bcde-5d62a7343b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:14.286380 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a85784-baba-47fd-bcde-5d62a7343b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:14.293127 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a85784-baba-47fd-bcde-5d62a7343b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:14.294740 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a85784-baba-47fd-bcde-5d62a7343b88 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:14.306824 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1459/2925] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:52:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:52:20.472272 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69fd93-3b9f-48b3-a977-3fa7a090f563 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:20.481806 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69fd93-3b9f-48b3-a977-3fa7a090f563 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:20.488554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb19d17-9607-40f5-b492-573085dd4440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0145242ef784168bada8348ccd95f22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:20.488554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb19d17-9607-40f5-b492-573085dd4440 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:20.488554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb19d17-9607-40f5-b492-573085dd4440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:20.488554 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb19d17-9607-40f5-b492-573085dd4440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:20.547788 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb19d17-9607-40f5-b492-573085dd4440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:20.549254 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb19d17-9607-40f5-b492-573085dd4440 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:20.559373 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1467/2926] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:52:23.919360 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a85784-baba-47fd-bcde-5d62a7343b88 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:23.932763 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a85784-baba-47fd-bcde-5d62a7343b88 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:23.938365 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4691ac8-5805-48a5-83c0-4e91dc5fc651 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:23.938985 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4691ac8-5805-48a5-83c0-4e91dc5fc651 service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:23.939330 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4691ac8-5805-48a5-83c0-4e91dc5fc651 service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:23.939604 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4691ac8-5805-48a5-83c0-4e91dc5fc651 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:23.946769 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a4691ac8-5805-48a5-83c0-4e91dc5fc651 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:23.959427 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4691ac8-5805-48a5-83c0-4e91dc5fc651 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:23.961619 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4691ac8-5805-48a5-83c0-4e91dc5fc651 service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:23.972804 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1460/2927] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:52:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:52:27.486774 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb19d17-9607-40f5-b492-573085dd4440 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:27.499031 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb19d17-9607-40f5-b492-573085dd4440 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:27.503807 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:27.504437 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:27.504719 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:27.504995 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] PATH_INFO: `/v3/users/4edfa99b10014e8289697a255d8b90cf` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:27.506206 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4edfa99b10014e8289697a255d8b90cf)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:27.508801 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:27.687816 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4edfa99b10014e8289697a255d8b90cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:27.688504 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4edfa99b10014e8289697a255d8b90cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:27.689096 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4edfa99b10014e8289697a255d8b90cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:27.695954 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4edfa99b10014e8289697a255d8b90cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:27.707894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4edfa99b10014e8289697a255d8b90cf'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:27.785401 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1468/2928] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:52:27 2022] DELETE /identity/v3/users/4edfa99b10014e8289697a255d8b90cf => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:52:28.041585 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4691ac8-5805-48a5-83c0-4e91dc5fc651 service placement] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:28.053133 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4691ac8-5805-48a5-83c0-4e91dc5fc651 service placement] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:28.113717 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:28.114377 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:28.114664 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:28.114940 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] PATH_INFO: `/v3/projects/548b5e4f59b24f0a9b88bf125c8a19a8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:28.116175 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=548b5e4f59b24f0a9b88bf125c8a19a8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:28.118775 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:28.223960 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '548b5e4f59b24f0a9b88bf125c8a19a8', 'request_id': 'req-18da6eff-d6fe-4f90-a82d-9e06569690c6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.229645 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '548b5e4f59b24f0a9b88bf125c8a19a8', 'request_id': 'req-18da6eff-d6fe-4f90-a82d-9e06569690c6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.236163 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '548b5e4f59b24f0a9b88bf125c8a19a8', 'request_id': 'req-18da6eff-d6fe-4f90-a82d-9e06569690c6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.310998 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] The token cache is being invalidate because project 548b5e4f59b24f0a9b88bf125c8a19a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:52:28.311219 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.311788 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.313294 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1461/2929] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:52:28 2022] DELETE /identity/v3/projects/548b5e4f59b24f0a9b88bf125c8a19a8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:52:28.317449 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:28.318896 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:28.332198 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfaee2-131d-4ab2-bf4b-307cd54d5563 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:28.380968 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:52:28.391660 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:28.392323 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:28.392628 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:28.392878 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] PATH_INFO: `/v3/users/79df3a80ae684517b81ddedc1de0a949` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:28.394107 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79df3a80ae684517b81ddedc1de0a949)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:28.396066 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:28.697845 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79df3a80ae684517b81ddedc1de0a949'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.698540 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79df3a80ae684517b81ddedc1de0a949'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.699145 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79df3a80ae684517b81ddedc1de0a949'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.706299 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79df3a80ae684517b81ddedc1de0a949'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.718752 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79df3a80ae684517b81ddedc1de0a949'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:28.833758 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1469/2930] 10.222.0.83 () {58 vars in 1224 bytes} [Wed Aug 24 11:52:28 2022] DELETE /identity/v3/users/79df3a80ae684517b81ddedc1de0a949 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:52:29.105194 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:29.106918 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:29.123482 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da6eff-d6fe-4f90-a82d-9e06569690c6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:29.182256 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d32a6d718b64e35aca00f733d966fd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17f9dd8b207b49df9eb8e79eece61c9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:29.182964 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:29.183521 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:29.183801 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] PATH_INFO: `/v3/projects/77647159ba75424ab303d44de5e7eef7` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:29.184949 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77647159ba75424ab303d44de5e7eef7)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:29.186980 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:29.193577 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:52:29.328224 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77647159ba75424ab303d44de5e7eef7', 'request_id': 'req-91966b40-5f15-4255-9f48-63c8a1ca7a55'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:29.333983 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77647159ba75424ab303d44de5e7eef7', 'request_id': 'req-91966b40-5f15-4255-9f48-63c8a1ca7a55'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:29.341352 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77647159ba75424ab303d44de5e7eef7', 'request_id': 'req-91966b40-5f15-4255-9f48-63c8a1ca7a55'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:29.459522 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] The token cache is being invalidate because project 77647159ba75424ab303d44de5e7eef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:52:29.459810 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:29.460450 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-91966b40-5f15-4255-9f48-63c8a1ca7a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:29.462078 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1462/2931] 10.222.0.83 () {58 vars in 1233 bytes} [Wed Aug 24 11:52:29 2022] DELETE /identity/v3/projects/77647159ba75424ab303d44de5e7eef7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:52:38.830093 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:52:39.491559 dv-cinder-854273-2-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:54:17.313524 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:54:17.314978 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:54:17.458416 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13fe2e9-44dd-4a10-ba09-8ff87c1fa820 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:54:17.523894 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba6d7b-7739-4231-a602-6032e9dd19f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb76ad667a744126ab00b38c346e9017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd7278f8c3f04cd391e6ffda9a5c8217', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:54:17.524748 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba6d7b-7739-4231-a602-6032e9dd19f9 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:54:17.525035 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba6d7b-7739-4231-a602-6032e9dd19f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:54:17.525486 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba6d7b-7739-4231-a602-6032e9dd19f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:54:17.528051 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-55ba6d7b-7739-4231-a602-6032e9dd19f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:54:17.541014 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba6d7b-7739-4231-a602-6032e9dd19f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:54:17.542650 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba6d7b-7739-4231-a602-6032e9dd19f9 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:54:17.553837 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1470/2932] 10.222.0.83 () {60 vars in 1365 bytes} [Wed Aug 24 11:54:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:54:27.550432 dv-cinder-854273-2-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}